Jul 22 18:56:37.609709 ip-10-0-137-176 systemd[1]: Starting Kubernetes Kubelet... Jul 22 18:56:39.577694 ip-10-0-137-176 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:56:39.577694 ip-10-0-137-176 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 18:56:39.577694 ip-10-0-137-176 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:56:39.577694 ip-10-0-137-176 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 18:56:39.577694 ip-10-0-137-176 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:56:39.578816 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.578644 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 18:56:39.586995 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.586967 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:56:39.586995 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.586991 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:56:39.586995 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.586995 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:56:39.586995 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.586998 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587001 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587005 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587008 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587011 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587014 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587020 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587024 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587028 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587046 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587049 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587052 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587055 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587058 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587061 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587064 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587067 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587070 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587072 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587076 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:56:39.587118 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587079 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587082 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587094 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587098 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587102 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587107 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587110 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587113 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587115 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587118 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587121 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587123 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587126 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587130 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587132 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587136 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587139 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587141 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587144 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587147 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:56:39.587580 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587149 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587151 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587154 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587156 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587159 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587161 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587164 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587166 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587168 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587171 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587173 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587176 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587179 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587182 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587187 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587190 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587192 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587195 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587197 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587200 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:56:39.588088 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587202 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587205 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587207 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587210 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587213 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587215 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587218 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587221 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587225 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587228 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587230 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587235 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587237 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587240 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587243 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587245 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587248 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587251 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587253 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:56:39.588563 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587256 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587259 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587261 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587264 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587266 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587269 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587272 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587275 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587278 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587280 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587283 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587285 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587288 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587291 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587293 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587296 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.587300 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588033 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588041 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:56:39.589034 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588044 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588047 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588051 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588061 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588064 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588068 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588071 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588074 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588076 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588079 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588082 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588084 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588087 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588090 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588094 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588098 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588102 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588105 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588108 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:56:39.589482 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588111 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588114 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588116 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588119 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588122 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588124 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588127 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588130 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588132 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588135 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588138 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588140 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588143 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588145 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588148 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588151 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588153 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588156 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588159 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588161 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:56:39.589956 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588164 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588166 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588169 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588172 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588174 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588177 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588179 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588182 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588184 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588187 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588190 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588193 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588196 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588199 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588201 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588204 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588206 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588209 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588211 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:56:39.590452 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588213 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588217 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588220 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588223 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588225 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588228 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588231 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588233 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588236 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588238 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588241 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588245 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588248 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588250 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588253 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588255 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588258 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588260 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588263 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:56:39.590930 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588265 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588268 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588272 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588274 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588277 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588280 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588283 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588285 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588288 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588290 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588293 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588296 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588298 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588300 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588303 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588306 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588308 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588311 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588314 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588316 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:56:39.591405 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588743 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588749 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588752 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588755 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588759 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588763 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588766 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588768 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588771 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588775 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588779 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588782 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588784 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588787 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588789 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588792 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588794 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588797 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588800 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:56:39.591919 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588803 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588807 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588810 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588813 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588815 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588818 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588820 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588823 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588826 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588828 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588830 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588833 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588835 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588838 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588840 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588843 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588847 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588851 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588854 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588857 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:56:39.592366 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588860 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588862 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588865 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588868 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588870 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588873 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588876 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588878 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588881 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588884 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588887 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588890 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588893 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588895 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588898 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588900 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588902 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588905 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588908 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588910 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:56:39.592903 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588913 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588915 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588918 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588920 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588923 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588925 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588928 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588931 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588935 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588937 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588940 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588943 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588946 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588949 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588952 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588955 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588958 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588961 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588964 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:56:39.593382 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588967 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588969 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588972 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588975 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588978 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588982 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588985 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588987 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588989 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588992 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588994 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.588997 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589000 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589002 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589004 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589007 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589009 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589012 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589014 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589017 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:56:39.593853 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589019 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589089 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589097 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589104 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589108 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589113 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589116 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589121 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589126 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589129 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589132 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589135 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589139 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589142 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589146 2565 flags.go:64] FLAG: --cgroup-root="" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589149 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589152 2565 flags.go:64] FLAG: --client-ca-file="" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589155 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589158 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589163 2565 flags.go:64] FLAG: --cluster-domain="" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589166 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589169 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589173 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589176 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 18:56:39.594352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589180 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589183 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589186 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589189 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589192 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589195 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589198 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589201 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589204 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589209 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589212 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589214 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589217 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589220 2565 flags.go:64] FLAG: --enable-server="true" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589224 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589229 2565 flags.go:64] FLAG: --event-burst="100" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589232 2565 flags.go:64] FLAG: --event-qps="50" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589235 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589238 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589241 2565 flags.go:64] FLAG: --eviction-hard="" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589245 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589248 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589252 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589255 2565 flags.go:64] FLAG: --eviction-soft="" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589261 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 18:56:39.594936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589264 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589267 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589270 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589273 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589276 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589279 2565 flags.go:64] FLAG: --feature-gates="" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589283 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589286 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589289 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589293 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589296 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589299 2565 flags.go:64] FLAG: --help="false" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589301 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589304 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589307 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589310 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589313 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589317 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589320 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589322 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589325 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589329 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589332 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589335 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 18:56:39.595554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589338 2565 flags.go:64] FLAG: --kube-reserved="" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589341 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589344 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589347 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589350 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589352 2565 flags.go:64] FLAG: --lock-file="" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589355 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589359 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589362 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589368 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589370 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589373 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589376 2565 flags.go:64] FLAG: --logging-format="text" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589379 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589382 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589385 2565 flags.go:64] FLAG: --manifest-url="" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589388 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589392 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589395 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589399 2565 flags.go:64] FLAG: --max-pods="110" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589402 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589405 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589408 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589411 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589413 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 18:56:39.596155 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589416 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589419 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589426 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589429 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589432 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589435 2565 flags.go:64] FLAG: --pod-cidr="" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589438 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589444 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589447 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589451 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589453 2565 flags.go:64] FLAG: --port="10250" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589456 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589459 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017e7e90e3fce8e7c" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589462 2565 flags.go:64] FLAG: --qos-reserved="" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589467 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589470 2565 flags.go:64] FLAG: --register-node="true" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589473 2565 flags.go:64] FLAG: --register-with-taints="" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589477 2565 flags.go:64] FLAG: --registry-burst="10" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589479 2565 flags.go:64] FLAG: --registry-qps="5" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589482 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589485 2565 flags.go:64] FLAG: --reserved-memory="" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589489 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589492 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589495 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589498 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 18:56:39.596775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589501 2565 flags.go:64] FLAG: --runonce="false" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589504 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589507 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589510 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589513 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589516 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589519 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589522 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589525 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589527 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589530 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589533 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589535 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589538 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589543 2565 flags.go:64] FLAG: --system-cgroups="" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589546 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589552 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589555 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589558 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589562 2565 flags.go:64] FLAG: --tls-min-version="" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589565 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589569 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589572 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589574 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589577 2565 flags.go:64] FLAG: --v="2" Jul 22 18:56:39.597389 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589581 2565 flags.go:64] FLAG: --version="false" Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589585 2565 flags.go:64] FLAG: --vmodule="" Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589589 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.589593 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589724 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589728 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589731 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589734 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589737 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589740 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589743 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589745 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589748 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589750 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589753 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589755 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589758 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589760 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589763 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589766 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:56:39.598070 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589768 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589772 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589775 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589778 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589781 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589783 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589786 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589788 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589792 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589795 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589798 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589800 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589803 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589805 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589807 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589810 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589813 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589816 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589819 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589821 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:56:39.598571 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589824 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589826 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589829 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589832 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589834 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589837 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589841 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589844 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589847 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589849 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589851 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589854 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589857 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589861 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589864 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589867 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589870 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589872 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589875 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589877 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:56:39.599074 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589881 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589885 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589887 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589889 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589892 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589894 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589897 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589900 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589902 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589905 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589907 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589910 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589913 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589916 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589918 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589922 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589925 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589928 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589931 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:56:39.599725 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589935 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589938 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589941 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589944 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589946 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589949 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589954 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589957 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589961 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589964 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589967 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589970 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589973 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589977 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589980 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589982 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589985 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589988 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589990 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589993 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:56:39.600422 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589995 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.589998 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.590000 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.590003 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.590008 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.597219 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.597238 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.597260 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.597266 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597324 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597331 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597336 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597340 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597343 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:56:39.600993 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597346 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597348 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597351 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597353 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597356 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597359 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597361 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597364 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597366 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597369 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597372 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597375 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597377 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597379 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597382 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597384 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597387 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597389 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597394 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:56:39.601351 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597398 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597400 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597403 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597406 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597409 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597413 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597416 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597418 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597421 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597423 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597426 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597428 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597431 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597433 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597436 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597438 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597441 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597443 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597446 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597448 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:56:39.601984 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597451 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597453 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597456 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597459 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597461 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597464 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597466 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597469 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597471 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597474 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597476 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597478 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597481 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597483 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597486 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597489 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597491 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597495 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597498 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597501 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:56:39.602522 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597503 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597506 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597508 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597511 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597514 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597516 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597519 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597521 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597523 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597526 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597528 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597531 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597533 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597536 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597539 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597542 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597544 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597547 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597549 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597551 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:56:39.603067 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597554 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597556 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597559 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597561 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597564 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597566 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597569 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597571 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597574 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597577 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597580 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597583 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597586 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597590 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597593 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:56:39.603567 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.597598 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597729 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597735 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597738 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597741 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597744 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597747 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597750 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597753 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597756 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597758 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597761 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597764 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597767 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597769 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597772 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597774 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597777 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597779 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597782 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:56:39.603967 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597784 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597787 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597789 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597792 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597794 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597797 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597800 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597803 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597805 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597808 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597811 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597814 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597816 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597819 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597821 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597824 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597827 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597829 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597832 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597834 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:56:39.604455 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597837 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597839 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597842 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597846 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597850 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597853 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597855 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597858 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597861 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597863 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597866 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597868 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597871 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597873 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597876 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597878 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597881 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597884 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597886 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597889 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:56:39.604959 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597892 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597894 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597898 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597900 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597903 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597905 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597908 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597910 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597914 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597917 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597920 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597922 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597925 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597927 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597930 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597932 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597935 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597937 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597940 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:56:39.605437 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597942 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597945 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597947 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597950 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597952 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597955 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597957 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597960 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597962 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597964 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597968 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597970 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597973 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597976 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597978 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597981 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597984 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597986 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597989 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:56:39.605907 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597991 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:39.597995 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.598000 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.598134 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.603601 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.605153 2565 server.go:1019] "Starting client certificate rotation" Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.605254 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 18:56:39.606362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.605291 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 18:56:39.607912 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.607899 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 18:56:39.609054 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.609035 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 18:56:39.617677 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.617658 2565 log.go:25] "Validated CRI v1 runtime API" Jul 22 18:56:39.621590 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.621574 2565 log.go:25] "Validated CRI v1 image API" Jul 22 18:56:39.623855 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.623829 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 18:56:39.626782 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.626760 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9a902f51-5855-413f-a13c-044be87beebb:/dev/nvme0n1p4 d24c79e3-7a14-4770-828a-87b04b348fb9:/dev/nvme0n1p3] Jul 22 18:56:39.626863 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.626781 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 18:56:39.632043 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.631932 2565 manager.go:216] Machine: {Timestamp:2026-07-22 18:56:39.630489536 +0000 UTC m=+0.357050628 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100079 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a84ad37c57794b16915751146b24f SystemUUID:ec2a84ad-37c5-7794-b169-15751146b24f BootID:fa9c4e0a-f8da-46d2-8f2c-431c16e5a470 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:70:ac:d0:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:70:ac:d0:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:a6:1a:ef:87:f1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 18:56:39.632043 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.632037 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 18:56:39.632160 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.632129 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 18:56:39.635268 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.635233 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 18:56:39.635453 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.635269 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-176.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 18:56:39.636549 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.636536 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 18:56:39.636549 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.636550 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 18:56:39.636645 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.636558 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 18:56:39.637205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.637194 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 18:56:39.638245 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.638229 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 18:56:39.638854 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.638843 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 18:56:39.642115 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642102 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 22 18:56:39.642235 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642226 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 18:56:39.642288 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642280 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 22 18:56:39.642318 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642305 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 18:56:39.642348 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642321 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 18:56:39.642348 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642330 2565 kubelet.go:398] "Adding apiserver pod source" Jul 22 18:56:39.642348 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642343 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 18:56:39.642876 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642851 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 18:56:39.642962 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.642888 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 18:56:39.647374 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.647338 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 18:56:39.649773 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.649755 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 18:56:39.649853 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.649787 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 18:56:39.650936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650921 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 18:56:39.650936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650942 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 18:56:39.650936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650948 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650954 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650963 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650969 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650977 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650982 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650989 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.650994 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.651003 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 18:56:39.651093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.651008 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 18:56:39.651792 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.651779 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 18:56:39.651827 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.651793 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 18:56:39.653455 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.653422 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jhwn7" Jul 22 18:56:39.653684 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.653656 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 18:56:39.653765 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.653707 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-176.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 18:56:39.653821 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.653805 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-176.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 18:56:39.655166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.655151 2565 server.go:1268] "Started kubelet" Jul 22 18:56:39.655241 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.655213 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 18:56:39.655435 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.655381 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 18:56:39.655563 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.655460 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 18:56:39.655965 ip-10-0-137-176 systemd[1]: Started Kubernetes Kubelet. Jul 22 18:56:39.657273 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.657255 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 18:56:39.657431 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.657417 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 22 18:56:39.660570 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.659521 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-176.ec2.internal.18c4b1938e8798cb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-176.ec2.internal,UID:ip-10-0-137-176.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-176.ec2.internal,},FirstTimestamp:2026-07-22 18:56:39.655119051 +0000 UTC m=+0.381680143,LastTimestamp:2026-07-22 18:56:39.655119051 +0000 UTC m=+0.381680143,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-176.ec2.internal,}" Jul 22 18:56:39.661140 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.661117 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 18:56:39.661195 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.661139 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jhwn7" Jul 22 18:56:39.666628 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.666599 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 18:56:39.666681 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.666634 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 18:56:39.666739 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.666703 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:39.667227 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667204 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 18:56:39.667227 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667225 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 18:56:39.667374 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667240 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 18:56:39.667598 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667574 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 22 18:56:39.667598 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667591 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 22 18:56:39.667820 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667806 2565 factory.go:153] Registering CRI-O factory Jul 22 18:56:39.667878 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667823 2565 factory.go:223] Registration of the crio container factory successfully Jul 22 18:56:39.667926 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667889 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 18:56:39.667926 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667909 2565 factory.go:55] Registering systemd factory Jul 22 18:56:39.667926 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667918 2565 factory.go:223] Registration of the systemd container factory successfully Jul 22 18:56:39.668030 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667940 2565 factory.go:103] Registering Raw factory Jul 22 18:56:39.668030 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.667973 2565 manager.go:1196] Started watching for new ooms in manager Jul 22 18:56:39.668461 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.668451 2565 manager.go:349] Starting recovery of all containers Jul 22 18:56:39.676803 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.676775 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:39.679977 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.679946 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-176.ec2.internal\" not found" node="ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.680089 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.680023 2565 manager.go:354] Recovery completed Jul 22 18:56:39.685249 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.685234 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.689247 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689228 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.689341 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689263 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.689341 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689275 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.689921 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689874 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 18:56:39.689921 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689888 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 18:56:39.689921 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689906 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 22 18:56:39.690069 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.689880 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.690736 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.690719 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.690821 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.690746 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.690821 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.690758 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.691379 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.691363 2565 policy_none.go:49] "None policy: Start" Jul 22 18:56:39.691379 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.691381 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 18:56:39.691474 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.691392 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 18:56:39.692316 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.692304 2565 policy_none.go:47] "Start" Jul 22 18:56:39.739540 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.739352 2565 manager.go:337] "Starting Device Plugin manager" Jul 22 18:56:39.739711 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.739594 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 18:56:39.739711 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.739624 2565 server.go:87] "Starting device plugin registration server" Jul 22 18:56:39.739892 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.739881 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 18:56:39.739930 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.739893 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 18:56:39.740040 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.740023 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 18:56:39.740129 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.740112 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 18:56:39.740129 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.740125 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 18:56:39.740622 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.740586 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 18:56:39.740696 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.740651 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:39.806121 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.806081 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 18:56:39.807307 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.807288 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 18:56:39.807364 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.807343 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 18:56:39.807403 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.807373 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 18:56:39.807451 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.807417 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 18:56:39.809795 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.809773 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:39.840680 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.840648 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.841925 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.841908 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.842014 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.841939 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.842014 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.841949 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.842014 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.841973 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.848089 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.848073 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.848158 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.848098 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-176.ec2.internal\": node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:39.864532 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.864503 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:39.907884 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.907849 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal"] Jul 22 18:56:39.907962 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.907948 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.908938 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.908925 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.909009 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.908953 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.909009 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.908963 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.910559 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.910546 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.910686 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.910671 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.910731 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.910714 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.911266 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.911250 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.911362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.911280 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.911362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.911292 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.911362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.911250 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.911362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.911341 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.911362 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.911350 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.912652 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.912636 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.912740 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.912658 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:56:39.913561 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.913544 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:56:39.913695 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.913577 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:56:39.913695 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.913594 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:56:39.947559 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.947489 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-176.ec2.internal\" not found" node="ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.952021 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.952004 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-176.ec2.internal\" not found" node="ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.965472 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:39.965451 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:39.968858 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.968838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3429a2d7cce5365bc3a8f8d7de0f04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal\" (UID: \"a3429a2d7cce5365bc3a8f8d7de0f04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.968956 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.968864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3429a2d7cce5365bc3a8f8d7de0f04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal\" (UID: \"a3429a2d7cce5365bc3a8f8d7de0f04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:39.968956 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:39.968883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10d7c501939fb84cb4667bd33ca0ad06-config\") pod \"kube-apiserver-proxy-ip-10-0-137-176.ec2.internal\" (UID: \"10d7c501939fb84cb4667bd33ca0ad06\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.066535 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.066495 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.069785 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.069757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3429a2d7cce5365bc3a8f8d7de0f04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal\" (UID: \"a3429a2d7cce5365bc3a8f8d7de0f04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.069912 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.069798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3429a2d7cce5365bc3a8f8d7de0f04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal\" (UID: \"a3429a2d7cce5365bc3a8f8d7de0f04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.069912 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.069818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10d7c501939fb84cb4667bd33ca0ad06-config\") pod \"kube-apiserver-proxy-ip-10-0-137-176.ec2.internal\" (UID: \"10d7c501939fb84cb4667bd33ca0ad06\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.069912 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.069761 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3429a2d7cce5365bc3a8f8d7de0f04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal\" (UID: \"a3429a2d7cce5365bc3a8f8d7de0f04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.069912 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.069856 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3429a2d7cce5365bc3a8f8d7de0f04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal\" (UID: \"a3429a2d7cce5365bc3a8f8d7de0f04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.069912 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.069843 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10d7c501939fb84cb4667bd33ca0ad06-config\") pod \"kube-apiserver-proxy-ip-10-0-137-176.ec2.internal\" (UID: \"10d7c501939fb84cb4667bd33ca0ad06\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.167210 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.167170 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.251662 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.251577 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.255684 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.255486 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.267353 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.267326 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.367691 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.367651 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.468136 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.468103 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.541006 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.540920 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:40.569163 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.569117 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.604445 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.604396 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 18:56:40.604899 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.604544 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 18:56:40.604899 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.604585 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 18:56:40.605017 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.604575 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 18:56:40.666379 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.666343 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 18:51:39 +0000 UTC" deadline="2028-01-30 04:15:01.000414038 +0000 UTC" Jul 22 18:56:40.666379 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.666379 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13353h18m20.334038419s" Jul 22 18:56:40.669362 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:40.669345 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-176.ec2.internal\" not found" Jul 22 18:56:40.670364 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.670350 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:40.767860 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.767835 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.771200 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:40.771173 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3429a2d7cce5365bc3a8f8d7de0f04a.slice/crio-eddf625a511b56a2f49f82b8f57eabd628798cd54ca04dadba941adfd7942247 WatchSource:0}: Error finding container eddf625a511b56a2f49f82b8f57eabd628798cd54ca04dadba941adfd7942247: Status 404 returned error can't find the container with id eddf625a511b56a2f49f82b8f57eabd628798cd54ca04dadba941adfd7942247 Jul 22 18:56:40.771433 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:40.771421 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d7c501939fb84cb4667bd33ca0ad06.slice/crio-db61b971e7ebb181974a110ec1afe35b40a5fac9833db06006aaf1f8694a6017 WatchSource:0}: Error finding container db61b971e7ebb181974a110ec1afe35b40a5fac9833db06006aaf1f8694a6017: Status 404 returned error can't find the container with id db61b971e7ebb181974a110ec1afe35b40a5fac9833db06006aaf1f8694a6017 Jul 22 18:56:40.774818 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.774800 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 18:56:40.776919 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.776904 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 18:56:40.777990 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.777976 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" Jul 22 18:56:40.786621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.786592 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 18:56:40.812210 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.812098 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" event={"ID":"10d7c501939fb84cb4667bd33ca0ad06","Type":"ContainerStarted","Data":"db61b971e7ebb181974a110ec1afe35b40a5fac9833db06006aaf1f8694a6017"} Jul 22 18:56:40.813166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:40.813143 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" event={"ID":"a3429a2d7cce5365bc3a8f8d7de0f04a","Type":"ContainerStarted","Data":"eddf625a511b56a2f49f82b8f57eabd628798cd54ca04dadba941adfd7942247"} Jul 22 18:56:41.643856 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.643821 2565 apiserver.go:52] "Watching apiserver" Jul 22 18:56:41.653549 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.653517 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 18:56:41.654959 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.654929 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal","openshift-cluster-node-tuning-operator/tuned-cpgbz","openshift-dns/node-resolver-c9786","openshift-image-registry/node-ca-mkx5n","openshift-multus/network-metrics-daemon-vnnnq","openshift-network-diagnostics/network-check-target-5dwdc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal","openshift-multus/multus-additional-cni-plugins-bt5vk","openshift-multus/multus-n52sf","openshift-network-operator/iptables-alerter-w8sbq","openshift-ovn-kubernetes/ovnkube-node-s9rsq","kube-system/global-pull-secret-syncer-5qpnv","kube-system/konnectivity-agent-n96vv"] Jul 22 18:56:41.656432 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.656408 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.657955 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.657935 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.658058 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.658018 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:41.658988 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.658971 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.659270 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.658975 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 18:56:41.659365 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.659250 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.659450 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.659426 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.659554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.659501 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mhszb\"" Jul 22 18:56:41.660195 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.660168 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.661296 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.661278 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5mc9d\"" Jul 22 18:56:41.661400 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.661300 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:41.661400 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.661357 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:41.661516 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.661483 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.661516 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.661488 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.662412 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.662395 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.662498 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.662471 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:41.662554 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.662526 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:41.662665 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.662646 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.662761 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.662744 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 18:56:41.662866 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.662849 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5ldsz\"" Jul 22 18:56:41.663670 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.663652 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.665055 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.665037 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.665781 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.665766 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 18:56:41.666321 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.666192 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 18:56:41.666321 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.666208 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.666321 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.666227 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.666321 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.666282 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lxdd9\"" Jul 22 18:56:41.666321 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.666304 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.666622 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.666509 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 18:56:41.667132 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.667113 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 18:56:41.667211 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.667159 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qtrcg\"" Jul 22 18:56:41.667698 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.667478 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.667698 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.667497 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 18:56:41.667698 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.667542 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 18:56:41.668780 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.668754 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8fc79\"" Jul 22 18:56:41.668870 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.668787 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.668870 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.668842 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.668981 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.668754 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.669813 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.669798 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7bcq5\"" Jul 22 18:56:41.670027 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.670007 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 18:56:41.670288 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.670272 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.671454 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.671437 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.671554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.671440 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8whx7\"" Jul 22 18:56:41.671554 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.671488 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.671830 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.671811 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 18:56:41.672563 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.672547 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 18:56:41.672842 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.672815 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 18:56:41.672842 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.672833 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bv2fg\"" Jul 22 18:56:41.673246 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.673227 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 18:56:41.673335 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.673274 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 18:56:41.673335 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.673304 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 18:56:41.673441 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.673408 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 18:56:41.677816 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.677791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-var-lib-kubelet\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.677918 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.677832 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:41.677918 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.677865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-modprobe-d\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.677918 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.677891 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-os-release\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.677940 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-etc-kubernetes\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-run\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9028fdbb-9f04-47f1-b412-60c902f3d6ac-tmp\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678067 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-daemon-config\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678097 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678132 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-k8s-cni-cncf-io\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678172 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-hostroot\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678213 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e083d91-f5fa-42ce-b755-872ff84c3af8-konnectivity-ca\") pod \"konnectivity-agent-n96vv\" (UID: \"2e083d91-f5fa-42ce-b755-872ff84c3af8\") " pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-cni-binary-copy\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxksx\" (UniqueName: \"kubernetes.io/projected/9028fdbb-9f04-47f1-b412-60c902f3d6ac-kube-api-access-rxksx\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-cni-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-sys-fs\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-system-cni-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6fqx\" (UniqueName: \"kubernetes.io/projected/6e4e1837-2baa-4c5d-927b-70682f840b7a-kube-api-access-z6fqx\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:41.678774 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e083d91-f5fa-42ce-b755-872ff84c3af8-agent-certs\") pod \"konnectivity-agent-n96vv\" (UID: \"2e083d91-f5fa-42ce-b755-872ff84c3af8\") " pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678652 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7f38ebc-93c7-4813-9b3f-3038502b046b-dbus\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678684 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wh6m\" (UniqueName: \"kubernetes.io/projected/c8da430a-be95-4881-9b8c-25b984a290e7-kube-api-access-5wh6m\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-device-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678760 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysctl-conf\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678792 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-lib-modules\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6655j\" (UniqueName: \"kubernetes.io/projected/e7a8e046-db0d-4d58-8fd5-991242108d47-kube-api-access-6655j\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.679475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.678878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.679835 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.679815 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-kubernetes\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.679918 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.679890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-cni-binary-copy\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.679972 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.679918 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-conf-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680025 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.679974 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:41.680025 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.680120 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680042 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-registration-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.680120 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8da430a-be95-4881-9b8c-25b984a290e7-hosts-file\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.680120 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-cnibin\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.680120 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-os-release\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.680306 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680147 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ae751a3-8d23-4553-ac23-3e752a6958b7-host\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.680306 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680172 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjfz6\" (UniqueName: \"kubernetes.io/projected/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-kube-api-access-sjfz6\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680306 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680195 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.680306 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680236 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7f38ebc-93c7-4813-9b3f-3038502b046b-kubelet-config\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.680306 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-sys\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.680503 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-kubelet\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680503 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680348 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-multus-certs\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680503 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680392 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-socket-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.680503 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680430 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysconfig\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.680503 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680468 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-host\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-socket-dir-parent\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680554 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-cni-bin\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-cnibin\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680640 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-netns\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680664 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ae751a3-8d23-4553-ac23-3e752a6958b7-serviceca\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680687 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtvz9\" (UniqueName: \"kubernetes.io/projected/8ae751a3-8d23-4553-ac23-3e752a6958b7-kube-api-access-qtvz9\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.680742 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680728 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-systemd\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-tuned\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8da430a-be95-4881-9b8c-25b984a290e7-tmp-dir\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysctl-d\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-cni-multus\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-system-cni-dir\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.681113 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.680984 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk52d\" (UniqueName: \"kubernetes.io/projected/7922174c-9acd-4037-afd8-d26f3745b099-kube-api-access-hk52d\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.684767 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.684747 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 18:56:41.703537 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.703514 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f5n2w" Jul 22 18:56:41.711009 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.710983 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f5n2w" Jul 22 18:56:41.768158 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.768125 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 18:56:41.781466 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-daemon-config\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781466 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-node-log\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.781722 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.781722 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-k8s-cni-cncf-io\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781722 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781593 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-k8s-cni-cncf-io\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781722 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-hostroot\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781722 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovnkube-script-lib\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.781722 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781720 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e083d91-f5fa-42ce-b755-872ff84c3af8-konnectivity-ca\") pod \"konnectivity-agent-n96vv\" (UID: \"2e083d91-f5fa-42ce-b755-872ff84c3af8\") " pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-cni-binary-copy\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781735 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-hostroot\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxksx\" (UniqueName: \"kubernetes.io/projected/9028fdbb-9f04-47f1-b412-60c902f3d6ac-kube-api-access-rxksx\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781772 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-cni-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781873 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wms2n\" (UniqueName: \"kubernetes.io/projected/a7776999-6853-453d-b2eb-852411b18170-kube-api-access-wms2n\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781904 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-cni-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781946 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-sys-fs\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.781971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-system-cni-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.781999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fqx\" (UniqueName: \"kubernetes.io/projected/6e4e1837-2baa-4c5d-927b-70682f840b7a-kube-api-access-z6fqx\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782017 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-cni-bin\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e083d91-f5fa-42ce-b755-872ff84c3af8-agent-certs\") pod \"konnectivity-agent-n96vv\" (UID: \"2e083d91-f5fa-42ce-b755-872ff84c3af8\") " pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782065 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-sys-fs\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782068 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-daemon-config\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782081 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-system-cni-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7f38ebc-93c7-4813-9b3f-3038502b046b-dbus\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782115 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wh6m\" (UniqueName: \"kubernetes.io/projected/c8da430a-be95-4881-9b8c-25b984a290e7-kube-api-access-5wh6m\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782133 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7f38ebc-93c7-4813-9b3f-3038502b046b-dbus\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-ovn\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782182 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-device-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysctl-conf\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782251 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-lib-modules\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782259 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e083d91-f5fa-42ce-b755-872ff84c3af8-konnectivity-ca\") pod \"konnectivity-agent-n96vv\" (UID: \"2e083d91-f5fa-42ce-b755-872ff84c3af8\") " pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.782397 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782279 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjjkm\" (UniqueName: \"kubernetes.io/projected/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-kube-api-access-bjjkm\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782280 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-device-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6655j\" (UniqueName: \"kubernetes.io/projected/e7a8e046-db0d-4d58-8fd5-991242108d47-kube-api-access-6655j\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782317 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-cni-binary-copy\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782393 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysctl-conf\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782394 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-kubernetes\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782407 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-lib-modules\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782499 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-cni-binary-copy\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782529 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-conf-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782546 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-kubernetes\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.782554 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782593 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-conf-dir\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.782650 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-env-overrides\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.783079 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.782715 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:56:42.282682313 +0000 UTC m=+3.009243393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.782801 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:56:42.282787143 +0000 UTC m=+3.009348235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovn-node-metrics-cert\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-registration-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782899 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8da430a-be95-4881-9b8c-25b984a290e7-hosts-file\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782920 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-run-netns\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782945 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-var-lib-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782952 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-cni-binary-copy\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-cnibin\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782975 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8da430a-be95-4881-9b8c-25b984a290e7-hosts-file\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782984 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-os-release\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.782984 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-registration-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ae751a3-8d23-4553-ac23-3e752a6958b7-host\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783020 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-cnibin\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjfz6\" (UniqueName: \"kubernetes.io/projected/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-kube-api-access-sjfz6\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.783896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783046 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-os-release\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783100 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ae751a3-8d23-4553-ac23-3e752a6958b7-host\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783146 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-systemd-units\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783202 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783209 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7f38ebc-93c7-4813-9b3f-3038502b046b-kubelet-config\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783233 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-sys\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-kubelet\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783285 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-multus-certs\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783293 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-sys\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783304 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-kubelet\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783310 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-kubelet\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783323 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7f38ebc-93c7-4813-9b3f-3038502b046b-kubelet-config\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783338 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-multus-certs\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-slash\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.784621 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783397 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7922174c-9acd-4037-afd8-d26f3745b099-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a7776999-6853-453d-b2eb-852411b18170-iptables-alerter-script\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-socket-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysconfig\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783485 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-host\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783508 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-socket-dir-parent\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783530 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-cni-bin\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783545 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7a8e046-db0d-4d58-8fd5-991242108d47-socket-dir\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-cnibin\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783563 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-host\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783579 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-multus-socket-dir-parent\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783598 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysconfig\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783627 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-cnibin\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783632 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-cni-bin\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-netns\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783668 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ae751a3-8d23-4553-ac23-3e752a6958b7-serviceca\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783693 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-run-netns\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.785205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtvz9\" (UniqueName: \"kubernetes.io/projected/8ae751a3-8d23-4553-ac23-3e752a6958b7-kube-api-access-qtvz9\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-systemd\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-tuned\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783902 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-log-socket\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783906 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-systemd\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783939 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovnkube-config\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8da430a-be95-4881-9b8c-25b984a290e7-tmp-dir\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.783986 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysctl-d\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-cni-multus\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-etc-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-cni-netd\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784125 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-host-var-lib-cni-multus\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-system-cni-dir\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784150 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-sysctl-d\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784162 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ae751a3-8d23-4553-ac23-3e752a6958b7-serviceca\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784194 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-system-cni-dir\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.786065 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk52d\" (UniqueName: \"kubernetes.io/projected/7922174c-9acd-4037-afd8-d26f3745b099-kube-api-access-hk52d\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784237 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8da430a-be95-4881-9b8c-25b984a290e7-tmp-dir\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784244 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-var-lib-kubelet\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-systemd\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-var-lib-kubelet\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784318 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7922174c-9acd-4037-afd8-d26f3745b099-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784323 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784349 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-modprobe-d\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784364 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-os-release\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-etc-kubernetes\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7776999-6853-453d-b2eb-852411b18170-host-slash\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784432 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-run\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784444 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-etc-kubernetes\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784454 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-os-release\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9028fdbb-9f04-47f1-b412-60c902f3d6ac-tmp\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784516 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-modprobe-d\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.784582 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9028fdbb-9f04-47f1-b412-60c902f3d6ac-run\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.785877 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e083d91-f5fa-42ce-b755-872ff84c3af8-agent-certs\") pod \"konnectivity-agent-n96vv\" (UID: \"2e083d91-f5fa-42ce-b755-872ff84c3af8\") " pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:41.786916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.786391 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9028fdbb-9f04-47f1-b412-60c902f3d6ac-tmp\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.787847 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.786564 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9028fdbb-9f04-47f1-b412-60c902f3d6ac-etc-tuned\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.792845 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.792822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxksx\" (UniqueName: \"kubernetes.io/projected/9028fdbb-9f04-47f1-b412-60c902f3d6ac-kube-api-access-rxksx\") pod \"tuned-cpgbz\" (UID: \"9028fdbb-9f04-47f1-b412-60c902f3d6ac\") " pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:41.795301 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.795269 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:56:41.795301 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.795301 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:56:41.795441 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.795315 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:41.795476 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:41.795443 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:42.29541597 +0000 UTC m=+3.021977073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:41.798205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.797029 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6655j\" (UniqueName: \"kubernetes.io/projected/e7a8e046-db0d-4d58-8fd5-991242108d47-kube-api-access-6655j\") pod \"aws-ebs-csi-driver-node-zrz6h\" (UID: \"e7a8e046-db0d-4d58-8fd5-991242108d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.799181 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.798720 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fqx\" (UniqueName: \"kubernetes.io/projected/6e4e1837-2baa-4c5d-927b-70682f840b7a-kube-api-access-z6fqx\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:41.799181 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.798763 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk52d\" (UniqueName: \"kubernetes.io/projected/7922174c-9acd-4037-afd8-d26f3745b099-kube-api-access-hk52d\") pod \"multus-additional-cni-plugins-bt5vk\" (UID: \"7922174c-9acd-4037-afd8-d26f3745b099\") " pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:41.799181 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.798806 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wh6m\" (UniqueName: \"kubernetes.io/projected/c8da430a-be95-4881-9b8c-25b984a290e7-kube-api-access-5wh6m\") pod \"node-resolver-c9786\" (UID: \"c8da430a-be95-4881-9b8c-25b984a290e7\") " pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.799181 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.799095 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjfz6\" (UniqueName: \"kubernetes.io/projected/3f4d5757-a5e8-4a05-bde9-4fe1897cfe74-kube-api-access-sjfz6\") pod \"multus-n52sf\" (UID: \"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74\") " pod="openshift-multus/multus-n52sf" Jul 22 18:56:41.799181 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.799141 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtvz9\" (UniqueName: \"kubernetes.io/projected/8ae751a3-8d23-4553-ac23-3e752a6958b7-kube-api-access-qtvz9\") pod \"node-ca-mkx5n\" (UID: \"8ae751a3-8d23-4553-ac23-3e752a6958b7\") " pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.885239 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885205 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885239 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885242 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-systemd-units\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885447 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885447 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885325 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-systemd-units\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885447 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885447 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885411 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885447 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885431 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-kubelet\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885447 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885446 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-slash\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a7776999-6853-453d-b2eb-852411b18170-iptables-alerter-script\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885519 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-kubelet\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-log-socket\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885530 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-slash\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885530 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-log-socket\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovnkube-config\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885701 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-etc-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.885725 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-cni-netd\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885744 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-etc-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-systemd\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885861 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-cni-netd\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885890 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-systemd\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7776999-6853-453d-b2eb-852411b18170-host-slash\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885926 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-node-log\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885952 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovnkube-script-lib\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885960 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7776999-6853-453d-b2eb-852411b18170-host-slash\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wms2n\" (UniqueName: \"kubernetes.io/projected/a7776999-6853-453d-b2eb-852411b18170-kube-api-access-wms2n\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.885990 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-node-log\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-cni-bin\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886040 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-ovn\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886067 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-cni-bin\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a7776999-6853-453d-b2eb-852411b18170-iptables-alerter-script\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjjkm\" (UniqueName: \"kubernetes.io/projected/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-kube-api-access-bjjkm\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886111 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-run-ovn\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886166 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-env-overrides\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovn-node-metrics-cert\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886224 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-run-netns\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886250 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-var-lib-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886249 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovnkube-config\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886295 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-var-lib-openvswitch\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886392 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-host-run-netns\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886524 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-env-overrides\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.886772 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.886535 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovnkube-script-lib\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.888620 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.888584 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-ovn-node-metrics-cert\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.897078 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.897017 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjjkm\" (UniqueName: \"kubernetes.io/projected/7253f10d-d34c-4e1c-81a4-fad95fd2e4ab-kube-api-access-bjjkm\") pod \"ovnkube-node-s9rsq\" (UID: \"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:41.897418 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.897396 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wms2n\" (UniqueName: \"kubernetes.io/projected/a7776999-6853-453d-b2eb-852411b18170-kube-api-access-wms2n\") pod \"iptables-alerter-w8sbq\" (UID: \"a7776999-6853-453d-b2eb-852411b18170\") " pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:41.973472 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.973436 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" Jul 22 18:56:41.982020 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.981991 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c9786" Jul 22 18:56:41.982964 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:41.982936 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a8e046_db0d_4d58_8fd5_991242108d47.slice/crio-dc2d9832e4ac3c97a320a8d1cad7db1e2af595b32ba45498ca354225df1188d8 WatchSource:0}: Error finding container dc2d9832e4ac3c97a320a8d1cad7db1e2af595b32ba45498ca354225df1188d8: Status 404 returned error can't find the container with id dc2d9832e4ac3c97a320a8d1cad7db1e2af595b32ba45498ca354225df1188d8 Jul 22 18:56:41.989895 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.989872 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mkx5n" Jul 22 18:56:41.996071 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:41.996048 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" Jul 22 18:56:42.002795 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.002776 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:56:42.004890 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:42.004860 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae751a3_8d23_4553_ac23_3e752a6958b7.slice/crio-9e5a44464820bf244f1dd8110e3564cd4a1135f3edb81841fc0eaef4d6ed1ec6 WatchSource:0}: Error finding container 9e5a44464820bf244f1dd8110e3564cd4a1135f3edb81841fc0eaef4d6ed1ec6: Status 404 returned error can't find the container with id 9e5a44464820bf244f1dd8110e3564cd4a1135f3edb81841fc0eaef4d6ed1ec6 Jul 22 18:56:42.008844 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:42.008818 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7922174c_9acd_4037_afd8_d26f3745b099.slice/crio-0437642c081879225e839912ab59e71bb4640b187d9c66e817e2d9964c178fd0 WatchSource:0}: Error finding container 0437642c081879225e839912ab59e71bb4640b187d9c66e817e2d9964c178fd0: Status 404 returned error can't find the container with id 0437642c081879225e839912ab59e71bb4640b187d9c66e817e2d9964c178fd0 Jul 22 18:56:42.008965 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.008944 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" Jul 22 18:56:42.016114 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.016091 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n52sf" Jul 22 18:56:42.023281 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.023192 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w8sbq" Jul 22 18:56:42.024697 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:42.024672 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e083d91_f5fa_42ce_b755_872ff84c3af8.slice/crio-00849b9782c633f84fd2ba6ee4befb4c242286b85356c4a1ec2b7f8d4779bb79 WatchSource:0}: Error finding container 00849b9782c633f84fd2ba6ee4befb4c242286b85356c4a1ec2b7f8d4779bb79: Status 404 returned error can't find the container with id 00849b9782c633f84fd2ba6ee4befb4c242286b85356c4a1ec2b7f8d4779bb79 Jul 22 18:56:42.026128 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:42.026065 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9028fdbb_9f04_47f1_b412_60c902f3d6ac.slice/crio-ba7307969b2b0085be096f7c003c52b28ea4fce3ea4e57001b929240e1f7cc92 WatchSource:0}: Error finding container ba7307969b2b0085be096f7c003c52b28ea4fce3ea4e57001b929240e1f7cc92: Status 404 returned error can't find the container with id ba7307969b2b0085be096f7c003c52b28ea4fce3ea4e57001b929240e1f7cc92 Jul 22 18:56:42.030060 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.029462 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:56:42.041565 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:42.041540 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7776999_6853_453d_b2eb_852411b18170.slice/crio-8e9b35b9ade4bd27e32b998358729d813f92945c98376400fd64ca8b3d66abe3 WatchSource:0}: Error finding container 8e9b35b9ade4bd27e32b998358729d813f92945c98376400fd64ca8b3d66abe3: Status 404 returned error can't find the container with id 8e9b35b9ade4bd27e32b998358729d813f92945c98376400fd64ca8b3d66abe3 Jul 22 18:56:42.047185 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:56:42.047147 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7253f10d_d34c_4e1c_81a4_fad95fd2e4ab.slice/crio-24c52cf48e484a168608294f723af6a6ac05ad811eb9f229851dc1bcb2595396 WatchSource:0}: Error finding container 24c52cf48e484a168608294f723af6a6ac05ad811eb9f229851dc1bcb2595396: Status 404 returned error can't find the container with id 24c52cf48e484a168608294f723af6a6ac05ad811eb9f229851dc1bcb2595396 Jul 22 18:56:42.161251 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.161169 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:42.184773 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.184729 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:42.289286 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.289247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:42.289446 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.289296 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:42.289446 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.289397 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:42.289446 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.289397 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:42.289583 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.289458 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:56:43.289441208 +0000 UTC m=+4.016002289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:42.289583 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.289473 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:56:43.289466892 +0000 UTC m=+4.016027971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:42.390518 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.390388 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:42.390723 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.390559 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:56:42.390723 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.390580 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:56:42.390723 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.390591 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:42.390723 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.390666 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:43.390647011 +0000 UTC m=+4.117208098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:42.685118 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.685083 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:56:42.713585 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.713544 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:51:41 +0000 UTC" deadline="2028-02-06 20:09:51.254424681 +0000 UTC" Jul 22 18:56:42.713585 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.713584 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13537h13m8.540845418s" Jul 22 18:56:42.807707 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.807646 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:42.807884 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.807780 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:42.808145 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.807977 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:42.808145 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:42.808106 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:42.819842 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.819803 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"24c52cf48e484a168608294f723af6a6ac05ad811eb9f229851dc1bcb2595396"} Jul 22 18:56:42.822316 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.822287 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w8sbq" event={"ID":"a7776999-6853-453d-b2eb-852411b18170","Type":"ContainerStarted","Data":"8e9b35b9ade4bd27e32b998358729d813f92945c98376400fd64ca8b3d66abe3"} Jul 22 18:56:42.824267 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.824229 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" event={"ID":"9028fdbb-9f04-47f1-b412-60c902f3d6ac","Type":"ContainerStarted","Data":"ba7307969b2b0085be096f7c003c52b28ea4fce3ea4e57001b929240e1f7cc92"} Jul 22 18:56:42.826185 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.826162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n96vv" event={"ID":"2e083d91-f5fa-42ce-b755-872ff84c3af8","Type":"ContainerStarted","Data":"00849b9782c633f84fd2ba6ee4befb4c242286b85356c4a1ec2b7f8d4779bb79"} Jul 22 18:56:42.828101 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.828033 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" event={"ID":"e7a8e046-db0d-4d58-8fd5-991242108d47","Type":"ContainerStarted","Data":"dc2d9832e4ac3c97a320a8d1cad7db1e2af595b32ba45498ca354225df1188d8"} Jul 22 18:56:42.830117 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.830096 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n52sf" event={"ID":"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74","Type":"ContainerStarted","Data":"1b89d959526ccfcf1ca675b6376eb43599496f2153624ea56b1e94aef2744a40"} Jul 22 18:56:42.831550 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.831527 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerStarted","Data":"0437642c081879225e839912ab59e71bb4640b187d9c66e817e2d9964c178fd0"} Jul 22 18:56:42.832826 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.832804 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mkx5n" event={"ID":"8ae751a3-8d23-4553-ac23-3e752a6958b7","Type":"ContainerStarted","Data":"9e5a44464820bf244f1dd8110e3564cd4a1135f3edb81841fc0eaef4d6ed1ec6"} Jul 22 18:56:42.834853 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.834828 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c9786" event={"ID":"c8da430a-be95-4881-9b8c-25b984a290e7","Type":"ContainerStarted","Data":"985f528abf000e0144b4fad6582911bd5fb196fb7b5e1f30b72aeaba2665336c"} Jul 22 18:56:42.839266 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.839236 2565 generic.go:356] "Generic (PLEG): container finished" podID="a3429a2d7cce5365bc3a8f8d7de0f04a" containerID="5591a0eecd8abdef7a221afae802264508ac2ec803172b0302178752754dcf19" exitCode=0 Jul 22 18:56:42.845988 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:42.845960 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" event={"ID":"a3429a2d7cce5365bc3a8f8d7de0f04a","Type":"ContainerDied","Data":"5591a0eecd8abdef7a221afae802264508ac2ec803172b0302178752754dcf19"} Jul 22 18:56:43.299506 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.298845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:43.299506 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.298896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:43.299506 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.299109 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:43.299506 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.299173 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:56:45.299153997 +0000 UTC m=+6.025715082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:43.299506 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.299356 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:43.299506 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.299401 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:56:45.299388488 +0000 UTC m=+6.025949573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:43.400258 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.399486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:43.400258 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.399753 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:56:43.400258 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.399775 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:56:43.400258 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.399788 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:43.400258 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.399849 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:45.399830113 +0000 UTC m=+6.126391198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:43.714468 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.714396 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:51:41 +0000 UTC" deadline="2028-03-12 07:23:14.622072658 +0000 UTC" Jul 22 18:56:43.714468 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.714464 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14364h26m30.907613334s" Jul 22 18:56:43.810053 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.810010 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:43.810240 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:43.810144 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:43.852820 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.852720 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" event={"ID":"10d7c501939fb84cb4667bd33ca0ad06","Type":"ContainerStarted","Data":"6aaac3a203ea7fc16f547c0ca5acfdcacc3568ff1a79ddedc2fed529d2abc5be"} Jul 22 18:56:43.869023 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.868204 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" event={"ID":"a3429a2d7cce5365bc3a8f8d7de0f04a","Type":"ContainerStarted","Data":"227da155cf893642a8b9d0a96a6a3065891d68015579d71dd30e154e462e806d"} Jul 22 18:56:43.869636 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.869573 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-176.ec2.internal" podStartSLOduration=3.869553305 podStartE2EDuration="3.869553305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:43.868896117 +0000 UTC m=+4.595457219" watchObservedRunningTime="2026-07-22 18:56:43.869553305 +0000 UTC m=+4.596114408" Jul 22 18:56:43.887168 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:43.887093 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-176.ec2.internal" podStartSLOduration=3.887064389 podStartE2EDuration="3.887064389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:43.886572537 +0000 UTC m=+4.613133640" watchObservedRunningTime="2026-07-22 18:56:43.887064389 +0000 UTC m=+4.613625491" Jul 22 18:56:44.808514 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:44.808482 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:44.808999 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:44.808639 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:44.809142 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:44.809087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:44.809221 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:44.809180 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:45.315694 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:45.314977 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:45.315694 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:45.315023 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:45.315694 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.315203 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:45.315694 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.315265 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:56:49.315247453 +0000 UTC m=+10.041808536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:45.315694 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.315455 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:45.315694 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.315518 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:56:49.315500927 +0000 UTC m=+10.042062349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:45.415896 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:45.415858 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:45.416088 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.416073 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:56:45.416155 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.416095 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:56:45.416155 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.416109 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:45.416252 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.416169 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:49.416148001 +0000 UTC m=+10.142709095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:45.808499 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:45.808420 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:45.808691 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:45.808556 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:46.808545 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:46.808502 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:46.808758 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:46.808676 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:46.808758 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:46.808715 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:46.809157 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:46.808805 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:47.808160 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:47.808105 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:47.808326 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:47.808236 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:48.807828 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:48.807735 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:48.807828 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:48.807783 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:48.808348 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:48.807877 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:48.808348 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:48.808027 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:49.350163 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:49.350121 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:49.350352 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:49.350172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:49.350352 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.350296 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:49.350352 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.350331 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:49.350539 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.350367 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:56:57.350347606 +0000 UTC m=+18.076908687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:49.350539 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.350401 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:56:57.350378765 +0000 UTC m=+18.076939845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:49.451187 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:49.451111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:49.451380 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.451280 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:56:49.451380 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.451303 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:56:49.451380 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.451315 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:49.451380 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.451379 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:57.451360152 +0000 UTC m=+18.177921243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:49.809273 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:49.809203 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:49.809703 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:49.809309 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:50.808536 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:50.808463 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:50.808728 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:50.808475 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:50.808728 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:50.808626 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:50.808728 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:50.808675 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:51.808627 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:51.808582 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:51.809100 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:51.808786 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:52.807795 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:52.807758 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:52.807960 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:52.807780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:52.807960 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:52.807889 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:52.808078 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:52.807979 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:53.807816 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:53.807778 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:53.808242 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:53.807906 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:54.808054 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:54.808017 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:54.808054 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:54.808045 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:54.808513 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:54.808141 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:54.808513 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:54.808314 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:55.808518 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:55.808488 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:55.808984 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:55.808620 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:56.808218 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:56.808183 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:56.808448 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:56.808294 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:56.808448 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:56.808351 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:56.808526 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:56.808442 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:57.406167 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:57.406131 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:57.406167 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:57.406171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:57.406774 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.406293 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:57.406774 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.406345 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:57:13.406330694 +0000 UTC m=+34.132891775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:56:57.406774 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.406294 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:57.406774 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.406420 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:57:13.406405956 +0000 UTC m=+34.132967037 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:56:57.506708 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:57.506668 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:57.506888 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.506794 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:56:57.506888 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.506815 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:56:57.506888 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.506828 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:57.506888 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.506889 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:13.506870248 +0000 UTC m=+34.233431330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:56:57.808128 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:57.808042 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:57.808291 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:57.808181 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:56:58.808616 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:58.808565 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:56:58.809050 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:58.808631 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:56:58.809050 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:58.808707 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:56:58.809050 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:58.808802 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:56:59.808422 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:56:59.808385 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:56:59.808592 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:56:59.808493 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:00.808188 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:00.808160 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:00.808542 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:00.808160 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:00.808542 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:00.808256 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:00.808542 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:00.808336 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:01.808916 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.808510 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:01.809733 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:01.809009 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:01.910471 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.910390 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"795fd78cbce660e4d311cffdee158bfd0e09f353208de244bfd577e099066bc8"} Jul 22 18:57:01.910471 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.910469 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"c5fa02b9053bf36cfd0f0b607387c698474f07488375ed494d1cc43df9250fbc"} Jul 22 18:57:01.910677 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.910508 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"0ca3ce22e0d985fe4d72123230705bc54953463fff4facc49b3459b91114dc0d"} Jul 22 18:57:01.910677 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.910545 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"0ccc0b734c7725f78185244e553f5cc971ef6caa25f558447293cc646386f7f7"} Jul 22 18:57:01.910677 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.910573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"e8252bc52a9a8305b879e51fbb2a90c94a616393cb99cbba67310c918d63d941"} Jul 22 18:57:01.910677 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.910635 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"aa120528e27ab4999a100582e8c9b9e1dec7c0c337e089721543e9123d1b1f34"} Jul 22 18:57:01.911886 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.911860 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" event={"ID":"9028fdbb-9f04-47f1-b412-60c902f3d6ac","Type":"ContainerStarted","Data":"fe7373a7dbe36825ad27dd4fec5345218ac72acdad98170a20b145a55b2f3b2a"} Jul 22 18:57:01.913431 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.913404 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n96vv" event={"ID":"2e083d91-f5fa-42ce-b755-872ff84c3af8","Type":"ContainerStarted","Data":"602500d23189b7510a56ef3df00031e1f3ee90828e51a932ceb44bd0ddd2aae5"} Jul 22 18:57:01.914765 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.914738 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" event={"ID":"e7a8e046-db0d-4d58-8fd5-991242108d47","Type":"ContainerStarted","Data":"7c54ce462c1cf1ec1f7cf15a0f8f57251409bf3aa2aed25325edff1bd95637f9"} Jul 22 18:57:01.916643 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.916617 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n52sf" event={"ID":"3f4d5757-a5e8-4a05-bde9-4fe1897cfe74","Type":"ContainerStarted","Data":"2b8e388a75587f163055621d2480faf6f7194cbabe8f852aabe89b4121f560a5"} Jul 22 18:57:01.918139 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.918115 2565 generic.go:356] "Generic (PLEG): container finished" podID="7922174c-9acd-4037-afd8-d26f3745b099" containerID="c510ee0998687190d4247811bcca886752ddc952a20cd47f943cd07fa507cb62" exitCode=0 Jul 22 18:57:01.918259 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.918199 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerDied","Data":"c510ee0998687190d4247811bcca886752ddc952a20cd47f943cd07fa507cb62"} Jul 22 18:57:01.919696 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.919676 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mkx5n" event={"ID":"8ae751a3-8d23-4553-ac23-3e752a6958b7","Type":"ContainerStarted","Data":"c5a4739af7e398135be43dce7d4f1856c21cfa216fdbd042de35b8ac265c0dc8"} Jul 22 18:57:01.921228 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.921130 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c9786" event={"ID":"c8da430a-be95-4881-9b8c-25b984a290e7","Type":"ContainerStarted","Data":"c6640cbb9eb790a1e2b1215fc16e950aab5bd271613fcee10d9090b65bd41592"} Jul 22 18:57:01.930227 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.930185 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cpgbz" podStartSLOduration=3.916644973 podStartE2EDuration="22.930169596s" totalImagesPullingTime="19.013524623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:01.92964328 +0000 UTC m=+22.656204378" watchObservedRunningTime="2026-07-22 18:57:01.930169596 +0000 UTC m=+22.656730701" Jul 22 18:57:01.964972 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.964918 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mkx5n" podStartSLOduration=3.963870217 podStartE2EDuration="22.96490423s" totalImagesPullingTime="19.001034013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:01.964337323 +0000 UTC m=+22.690898428" watchObservedRunningTime="2026-07-22 18:57:01.96490423 +0000 UTC m=+22.691465333" Jul 22 18:57:01.983484 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:01.983435 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n96vv" podStartSLOduration=8.165072485 podStartE2EDuration="22.983415762s" totalImagesPullingTime="14.818343277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:01.983208421 +0000 UTC m=+22.709769514" watchObservedRunningTime="2026-07-22 18:57:01.983415762 +0000 UTC m=+22.709976932" Jul 22 18:57:02.006282 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.006233 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n52sf" podStartSLOduration=3.751283204 podStartE2EDuration="23.006219801s" totalImagesPullingTime="19.254936597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:02.005838069 +0000 UTC m=+22.732399172" watchObservedRunningTime="2026-07-22 18:57:02.006219801 +0000 UTC m=+22.732780903" Jul 22 18:57:02.028108 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.028048 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-c9786" podStartSLOduration=4.017262939 podStartE2EDuration="23.028029776s" totalImagesPullingTime="19.010766837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:02.02766158 +0000 UTC m=+22.754222683" watchObservedRunningTime="2026-07-22 18:57:02.028029776 +0000 UTC m=+22.754590878" Jul 22 18:57:02.563602 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.563570 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 18:57:02.755098 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.754913 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T18:57:02.563595036Z","UUID":"379be268-e002-41c5-a9a1-0d148c6f268c","Handler":null,"Name":"","Endpoint":""} Jul 22 18:57:02.756950 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.756930 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 18:57:02.757090 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.756967 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 18:57:02.808093 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.808052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:02.808291 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.808071 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:02.808291 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:02.808238 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:02.808291 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:02.808274 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:02.925214 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.925128 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w8sbq" event={"ID":"a7776999-6853-453d-b2eb-852411b18170","Type":"ContainerStarted","Data":"cefcb73680811d8f43fbec026409217280827d80e7cd906a123c8facc423578c"} Jul 22 18:57:02.927543 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.927513 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" event={"ID":"e7a8e046-db0d-4d58-8fd5-991242108d47","Type":"ContainerStarted","Data":"463f9b928bada40c656ce3af5882d8d6bd88a492cb03f151acc0ef2263a7ce99"} Jul 22 18:57:02.941980 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:02.941933 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w8sbq" podStartSLOduration=4.978006848 podStartE2EDuration="23.941914787s" totalImagesPullingTime="18.963907939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:02.941254031 +0000 UTC m=+23.667815134" watchObservedRunningTime="2026-07-22 18:57:02.941914787 +0000 UTC m=+23.668475890" Jul 22 18:57:03.726162 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.726118 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:57:03.726871 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.726837 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:57:03.807982 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.807953 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:03.808170 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:03.808082 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:03.933975 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.933694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"047998fe767492b8d85d1efce8af0d13e1ffe47a7671a2eb7861843d8c0ba86f"} Jul 22 18:57:03.936828 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.936584 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" event={"ID":"e7a8e046-db0d-4d58-8fd5-991242108d47","Type":"ContainerStarted","Data":"a88702aa83f2574d9c9236aa400d64d23f9aed5734146e8d72979ff19b9ab66f"} Jul 22 18:57:03.937407 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.937317 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:57:03.937566 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.937547 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n96vv" Jul 22 18:57:03.954701 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:03.954653 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrz6h" podStartSLOduration=3.747283601 podStartE2EDuration="24.954637811s" totalImagesPullingTime="21.20735421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:03.95431132 +0000 UTC m=+24.680872421" watchObservedRunningTime="2026-07-22 18:57:03.954637811 +0000 UTC m=+24.681198912" Jul 22 18:57:04.807833 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:04.807799 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:04.808006 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:04.807948 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:04.808006 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:04.807977 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:04.808117 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:04.808090 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:05.808433 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:05.808392 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:05.808966 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:05.808520 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:06.808674 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.808486 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:06.809313 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.808484 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:06.809313 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:06.808757 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:06.809313 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:06.808806 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:06.946388 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.946351 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" event={"ID":"7253f10d-d34c-4e1c-81a4-fad95fd2e4ab","Type":"ContainerStarted","Data":"0ddec02c5a9946a0379b7912b6b36336abcd27efb26b4e909a57fb2642f696e8"} Jul 22 18:57:06.946696 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.946672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:57:06.948208 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.948182 2565 generic.go:356] "Generic (PLEG): container finished" podID="7922174c-9acd-4037-afd8-d26f3745b099" containerID="7c91f11ebff15c020ef9656962db80fd892e8cb84095afa949ae413bae54857c" exitCode=0 Jul 22 18:57:06.948304 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.948225 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerDied","Data":"7c91f11ebff15c020ef9656962db80fd892e8cb84095afa949ae413bae54857c"} Jul 22 18:57:06.964402 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.964376 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:57:06.975083 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:06.975033 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" podStartSLOduration=8.981863424 podStartE2EDuration="27.975020035s" totalImagesPullingTime="18.993156611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:06.973562326 +0000 UTC m=+27.700123428" watchObservedRunningTime="2026-07-22 18:57:06.975020035 +0000 UTC m=+27.701581137" Jul 22 18:57:07.808423 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:07.808352 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:07.808665 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:07.808456 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:07.951779 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:07.951743 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:57:07.952244 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:07.951852 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:57:07.971050 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:07.971022 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:57:08.232127 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.232094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qpnv"] Jul 22 18:57:08.232379 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.232214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:08.232379 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:08.232332 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:08.236815 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.236787 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vnnnq"] Jul 22 18:57:08.236957 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.236922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:08.237063 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:08.237040 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:08.237485 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.237465 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5dwdc"] Jul 22 18:57:08.237575 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.237536 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:08.237658 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:08.237640 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:08.955496 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.955260 2565 generic.go:356] "Generic (PLEG): container finished" podID="7922174c-9acd-4037-afd8-d26f3745b099" containerID="4f7abe935d74388ba8717bfeb1986a2e05aea094e36ea219cdaf85f9bd13e913" exitCode=0 Jul 22 18:57:08.955496 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:08.955337 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerDied","Data":"4f7abe935d74388ba8717bfeb1986a2e05aea094e36ea219cdaf85f9bd13e913"} Jul 22 18:57:09.811306 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:09.811272 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:09.811528 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:09.811363 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:09.811528 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:09.811371 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:09.811528 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:09.811388 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:09.811528 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:09.811470 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:09.811528 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:09.811528 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:10.962573 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:10.962541 2565 generic.go:356] "Generic (PLEG): container finished" podID="7922174c-9acd-4037-afd8-d26f3745b099" containerID="879c090370f515acf23b6d3ebac1a4e85162a3867f8bbe17b2939d01825f833c" exitCode=0 Jul 22 18:57:10.963125 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:10.962602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerDied","Data":"879c090370f515acf23b6d3ebac1a4e85162a3867f8bbe17b2939d01825f833c"} Jul 22 18:57:11.811132 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:11.811101 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:11.811313 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:11.811145 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:11.811313 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:11.811203 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:11.811313 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:11.811216 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:11.811313 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:11.811262 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:11.811534 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:11.811348 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:13.419325 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:13.419288 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:13.419956 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:13.419336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:13.419956 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.419462 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:57:13.419956 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.419524 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs podName:6e4e1837-2baa-4c5d-927b-70682f840b7a nodeName:}" failed. No retries permitted until 2026-07-22 18:57:45.419504534 +0000 UTC m=+66.146065627 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs") pod "network-metrics-daemon-vnnnq" (UID: "6e4e1837-2baa-4c5d-927b-70682f840b7a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:57:13.419956 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.419463 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:57:13.419956 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.419621 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret podName:e7f38ebc-93c7-4813-9b3f-3038502b046b nodeName:}" failed. No retries permitted until 2026-07-22 18:57:45.419588166 +0000 UTC m=+66.146149251 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret") pod "global-pull-secret-syncer-5qpnv" (UID: "e7f38ebc-93c7-4813-9b3f-3038502b046b") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:57:13.519971 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:13.519937 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:13.520128 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.520091 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:57:13.520128 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.520107 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:57:13.520128 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.520119 2565 projected.go:196] Error preparing data for projected volume kube-api-access-f7x65 for pod openshift-network-diagnostics/network-check-target-5dwdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:57:13.520250 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.520168 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65 podName:138b9c50-9adc-41a8-af93-38a9a81982c1 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:45.520154044 +0000 UTC m=+66.246715141 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7x65" (UniqueName: "kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65") pod "network-check-target-5dwdc" (UID: "138b9c50-9adc-41a8-af93-38a9a81982c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:57:13.811145 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:13.811052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:13.811292 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:13.811209 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:13.811292 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.811216 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qpnv" podUID="e7f38ebc-93c7-4813-9b3f-3038502b046b" Jul 22 18:57:13.811381 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:13.811214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:13.811381 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.811339 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnnnq" podUID="6e4e1837-2baa-4c5d-927b-70682f840b7a" Jul 22 18:57:13.811468 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:13.811391 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dwdc" podUID="138b9c50-9adc-41a8-af93-38a9a81982c1" Jul 22 18:57:15.112817 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.112795 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-176.ec2.internal" event="NodeReady" Jul 22 18:57:15.113263 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.113233 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 18:57:15.174445 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.174401 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dfkvn"] Jul 22 18:57:15.177070 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.177017 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6x589"] Jul 22 18:57:15.177205 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.177142 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.178999 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.178976 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.180211 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.180190 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cc98d\"" Jul 22 18:57:15.180826 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.180810 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 18:57:15.180910 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.180809 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 18:57:15.181472 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.181436 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 18:57:15.181588 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.181572 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dm4b7\"" Jul 22 18:57:15.181680 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.181579 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 18:57:15.181680 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.181636 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 18:57:15.193522 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.192793 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6x589"] Jul 22 18:57:15.193895 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.193877 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dfkvn"] Jul 22 18:57:15.329985 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.329901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czkk\" (UniqueName: \"kubernetes.io/projected/7578bff6-22d9-4bd1-981f-0fccb3b3bf8a-kube-api-access-5czkk\") pod \"ingress-canary-6x589\" (UID: \"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a\") " pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.329985 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.329948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.329985 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.329986 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-tmp-dir\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.330212 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.330003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ktf\" (UniqueName: \"kubernetes.io/projected/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-kube-api-access-27ktf\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.330212 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.330019 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7578bff6-22d9-4bd1-981f-0fccb3b3bf8a-cert\") pod \"ingress-canary-6x589\" (UID: \"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a\") " pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.330212 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.330036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-config-volume\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.431329 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431300 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-tmp-dir\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.431410 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431335 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ktf\" (UniqueName: \"kubernetes.io/projected/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-kube-api-access-27ktf\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.431410 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7578bff6-22d9-4bd1-981f-0fccb3b3bf8a-cert\") pod \"ingress-canary-6x589\" (UID: \"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a\") " pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.431410 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-config-volume\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.431516 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czkk\" (UniqueName: \"kubernetes.io/projected/7578bff6-22d9-4bd1-981f-0fccb3b3bf8a-kube-api-access-5czkk\") pod \"ingress-canary-6x589\" (UID: \"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a\") " pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.431516 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431457 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.431586 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:15.431551 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:57:15.431653 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:15.431641 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls podName:a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb nodeName:}" failed. No retries permitted until 2026-07-22 18:57:15.931600777 +0000 UTC m=+36.658161857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls") pod "dns-default-dfkvn" (UID: "a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb") : secret "dns-default-metrics-tls" not found Jul 22 18:57:15.431827 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431778 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-tmp-dir\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.431935 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.431917 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-config-volume\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.435726 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.435580 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7578bff6-22d9-4bd1-981f-0fccb3b3bf8a-cert\") pod \"ingress-canary-6x589\" (UID: \"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a\") " pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.442017 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.441992 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ktf\" (UniqueName: \"kubernetes.io/projected/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-kube-api-access-27ktf\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.442393 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.442370 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czkk\" (UniqueName: \"kubernetes.io/projected/7578bff6-22d9-4bd1-981f-0fccb3b3bf8a-kube-api-access-5czkk\") pod \"ingress-canary-6x589\" (UID: \"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a\") " pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.494194 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.494152 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6x589" Jul 22 18:57:15.624266 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.624167 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6x589"] Jul 22 18:57:15.628109 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:15.628083 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7578bff6_22d9_4bd1_981f_0fccb3b3bf8a.slice/crio-688bbadc5d5889f5e2d9fbc582e60e3afc6bae8fdfe21cd539bcd21e28cf6972 WatchSource:0}: Error finding container 688bbadc5d5889f5e2d9fbc582e60e3afc6bae8fdfe21cd539bcd21e28cf6972: Status 404 returned error can't find the container with id 688bbadc5d5889f5e2d9fbc582e60e3afc6bae8fdfe21cd539bcd21e28cf6972 Jul 22 18:57:15.807992 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.807958 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:15.808136 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.807957 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:15.808223 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.807968 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:15.810633 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.810595 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5knzw\"" Jul 22 18:57:15.810951 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.810933 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g8nmg\"" Jul 22 18:57:15.811042 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.810992 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 18:57:15.811042 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.810998 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 18:57:15.811710 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.811691 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 18:57:15.811799 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.811713 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 18:57:15.935404 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.935373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:15.935582 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:15.935484 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:57:15.935582 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:15.935549 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls podName:a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb nodeName:}" failed. No retries permitted until 2026-07-22 18:57:16.935529899 +0000 UTC m=+37.662090980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls") pod "dns-default-dfkvn" (UID: "a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb") : secret "dns-default-metrics-tls" not found Jul 22 18:57:15.977207 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.977174 2565 generic.go:356] "Generic (PLEG): container finished" podID="7922174c-9acd-4037-afd8-d26f3745b099" containerID="65ff2c7a9a7cc1ab94b997cc4a72238c194a83e0537209fa5685bfa865d3b605" exitCode=0 Jul 22 18:57:15.977374 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.977252 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerDied","Data":"65ff2c7a9a7cc1ab94b997cc4a72238c194a83e0537209fa5685bfa865d3b605"} Jul 22 18:57:15.978455 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:15.978436 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6x589" event={"ID":"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a","Type":"ContainerStarted","Data":"688bbadc5d5889f5e2d9fbc582e60e3afc6bae8fdfe21cd539bcd21e28cf6972"} Jul 22 18:57:16.944468 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:16.944425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:16.944847 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:16.944652 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:57:16.944847 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:16.944742 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls podName:a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb nodeName:}" failed. No retries permitted until 2026-07-22 18:57:18.944719352 +0000 UTC m=+39.671280448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls") pod "dns-default-dfkvn" (UID: "a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb") : secret "dns-default-metrics-tls" not found Jul 22 18:57:16.983993 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:16.983956 2565 generic.go:356] "Generic (PLEG): container finished" podID="7922174c-9acd-4037-afd8-d26f3745b099" containerID="67252e4c5c71301158ba0a22e7b6d6fc2b944e80954ac212decaf36b0ee29ad9" exitCode=0 Jul 22 18:57:16.984139 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:16.984013 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerDied","Data":"67252e4c5c71301158ba0a22e7b6d6fc2b944e80954ac212decaf36b0ee29ad9"} Jul 22 18:57:17.989686 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:17.989478 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" event={"ID":"7922174c-9acd-4037-afd8-d26f3745b099","Type":"ContainerStarted","Data":"e114977a93033dab9b24dd9d8f8f0bcd122e26b0e5fa3a255d1327fc34ce55ba"} Jul 22 18:57:18.022980 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:18.022923 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bt5vk" podStartSLOduration=9.181770768 podStartE2EDuration="39.022906401s" totalImagesPullingTime="29.841135633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:18.021718924 +0000 UTC m=+38.748280025" watchObservedRunningTime="2026-07-22 18:57:18.022906401 +0000 UTC m=+38.749467503" Jul 22 18:57:18.963553 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:18.963520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:18.963721 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:18.963642 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:57:18.963721 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:18.963696 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls podName:a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb nodeName:}" failed. No retries permitted until 2026-07-22 18:57:22.963681051 +0000 UTC m=+43.690242134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls") pod "dns-default-dfkvn" (UID: "a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb") : secret "dns-default-metrics-tls" not found Jul 22 18:57:18.993748 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:18.993714 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6x589" event={"ID":"7578bff6-22d9-4bd1-981f-0fccb3b3bf8a","Type":"ContainerStarted","Data":"906df90575323a356e404de2a2b04a780d4f4dd382440b5f43ca37649f491707"} Jul 22 18:57:19.009701 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:19.009649 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6x589" podStartSLOduration=1.535288669 podStartE2EDuration="4.009631586s" totalImagesPullingTime="2.474342917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:19.009348246 +0000 UTC m=+39.735909349" watchObservedRunningTime="2026-07-22 18:57:19.009631586 +0000 UTC m=+39.736192667" Jul 22 18:57:22.993569 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:22.993524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:22.993949 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:22.993691 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:57:22.993949 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:22.993749 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls podName:a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb nodeName:}" failed. No retries permitted until 2026-07-22 18:57:30.993732086 +0000 UTC m=+51.720293169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls") pod "dns-default-dfkvn" (UID: "a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb") : secret "dns-default-metrics-tls" not found Jul 22 18:57:24.429038 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.429005 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-flr6f"] Jul 22 18:57:24.459360 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.459328 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-flr6f"] Jul 22 18:57:24.459508 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.459458 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.461845 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.461815 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 18:57:24.461976 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.461870 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 18:57:24.462727 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.462711 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 18:57:24.462813 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.462726 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wspht\"" Jul 22 18:57:24.462813 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.462715 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 18:57:24.605325 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.605286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6859a523-ead0-4896-bd3a-826f9998e504-data-volume\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.605524 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.605349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6w2x\" (UniqueName: \"kubernetes.io/projected/6859a523-ead0-4896-bd3a-826f9998e504-kube-api-access-l6w2x\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.605524 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.605405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6859a523-ead0-4896-bd3a-826f9998e504-crio-socket\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.605524 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.605439 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.605524 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.605485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.705994 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.705910 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.705994 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.705971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6859a523-ead0-4896-bd3a-826f9998e504-data-volume\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.706188 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.706020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6w2x\" (UniqueName: \"kubernetes.io/projected/6859a523-ead0-4896-bd3a-826f9998e504-kube-api-access-l6w2x\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.706188 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.706046 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6859a523-ead0-4896-bd3a-826f9998e504-crio-socket\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.706188 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:24.706068 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:57:24.706188 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:24.706138 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:57:24.706188 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:24.706144 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:25.206120749 +0000 UTC m=+45.932681829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : secret "insights-runtime-extractor-tls" not found Jul 22 18:57:24.706188 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.706072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.706393 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:24.706210 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:25.206193807 +0000 UTC m=+45.932754913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : configmap "kube-rbac-proxy" not found Jul 22 18:57:24.706393 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.706264 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6859a523-ead0-4896-bd3a-826f9998e504-crio-socket\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.706393 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.706322 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6859a523-ead0-4896-bd3a-826f9998e504-data-volume\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.715747 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.715723 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6w2x\" (UniqueName: \"kubernetes.io/projected/6859a523-ead0-4896-bd3a-826f9998e504-kube-api-access-l6w2x\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:24.828122 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.828088 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv"] Jul 22 18:57:24.848752 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.848716 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv"] Jul 22 18:57:24.848894 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.848814 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" Jul 22 18:57:24.852958 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.852937 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-f2qxg\"" Jul 22 18:57:24.853276 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.853254 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 18:57:24.853384 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:24.853265 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 18:57:25.009312 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.009229 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tlj\" (UniqueName: \"kubernetes.io/projected/b45ec387-cde4-4b3e-9ad6-742c19bef34c-kube-api-access-s8tlj\") pod \"migrator-7cc9c7ffd7-t4bjv\" (UID: \"b45ec387-cde4-4b3e-9ad6-742c19bef34c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" Jul 22 18:57:25.109705 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.109662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tlj\" (UniqueName: \"kubernetes.io/projected/b45ec387-cde4-4b3e-9ad6-742c19bef34c-kube-api-access-s8tlj\") pod \"migrator-7cc9c7ffd7-t4bjv\" (UID: \"b45ec387-cde4-4b3e-9ad6-742c19bef34c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" Jul 22 18:57:25.117323 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.117289 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tlj\" (UniqueName: \"kubernetes.io/projected/b45ec387-cde4-4b3e-9ad6-742c19bef34c-kube-api-access-s8tlj\") pod \"migrator-7cc9c7ffd7-t4bjv\" (UID: \"b45ec387-cde4-4b3e-9ad6-742c19bef34c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" Jul 22 18:57:25.157717 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.157682 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" Jul 22 18:57:25.210881 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.210841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:25.211047 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.210936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:25.211047 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:25.210939 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:57:25.211047 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:25.211020 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:26.210998649 +0000 UTC m=+46.937559754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : configmap "kube-rbac-proxy" not found Jul 22 18:57:25.211233 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:25.211057 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:57:25.211233 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:25.211129 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:26.211116335 +0000 UTC m=+46.937677416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : secret "insights-runtime-extractor-tls" not found Jul 22 18:57:25.289394 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:25.289283 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv"] Jul 22 18:57:25.291947 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:25.291922 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45ec387_cde4_4b3e_9ad6_742c19bef34c.slice/crio-0c43312f2c249a3bc07983f6f7a354aa4b420dd5d0e566dc18b36c4d3b08a887 WatchSource:0}: Error finding container 0c43312f2c249a3bc07983f6f7a354aa4b420dd5d0e566dc18b36c4d3b08a887: Status 404 returned error can't find the container with id 0c43312f2c249a3bc07983f6f7a354aa4b420dd5d0e566dc18b36c4d3b08a887 Jul 22 18:57:26.010622 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:26.010571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" event={"ID":"b45ec387-cde4-4b3e-9ad6-742c19bef34c","Type":"ContainerStarted","Data":"0c43312f2c249a3bc07983f6f7a354aa4b420dd5d0e566dc18b36c4d3b08a887"} Jul 22 18:57:26.219732 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:26.219688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:26.219901 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:26.219776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:26.219901 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:26.219856 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:57:26.219901 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:26.219859 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:57:26.220004 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:26.219917 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:28.219903976 +0000 UTC m=+48.946465055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : configmap "kube-rbac-proxy" not found Jul 22 18:57:26.220004 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:26.219931 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:28.21992503 +0000 UTC m=+48.946486110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : secret "insights-runtime-extractor-tls" not found Jul 22 18:57:26.412972 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:26.412946 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c9786_c8da430a-be95-4881-9b8c-25b984a290e7/dns-node-resolver/0.log" Jul 22 18:57:27.016201 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.016172 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fbkxp"] Jul 22 18:57:27.031641 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.031598 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fbkxp"] Jul 22 18:57:27.031783 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.031711 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.034451 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.034431 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 18:57:27.034596 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.034550 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 18:57:27.034684 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.034622 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 18:57:27.035406 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.035390 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 18:57:27.035406 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.035402 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mqlmm\"" Jul 22 18:57:27.128224 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.128186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1491c729-3b1e-4f63-a65b-e64308d91c27-signing-key\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.128224 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.128222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1491c729-3b1e-4f63-a65b-e64308d91c27-signing-cabundle\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.128436 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.128352 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vkw4\" (UniqueName: \"kubernetes.io/projected/1491c729-3b1e-4f63-a65b-e64308d91c27-kube-api-access-7vkw4\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.229230 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.229196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vkw4\" (UniqueName: \"kubernetes.io/projected/1491c729-3b1e-4f63-a65b-e64308d91c27-kube-api-access-7vkw4\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.229396 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.229266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1491c729-3b1e-4f63-a65b-e64308d91c27-signing-key\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.229396 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.229284 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1491c729-3b1e-4f63-a65b-e64308d91c27-signing-cabundle\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.229975 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.229958 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1491c729-3b1e-4f63-a65b-e64308d91c27-signing-cabundle\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.231530 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.231511 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1491c729-3b1e-4f63-a65b-e64308d91c27-signing-key\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.237364 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.237343 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vkw4\" (UniqueName: \"kubernetes.io/projected/1491c729-3b1e-4f63-a65b-e64308d91c27-kube-api-access-7vkw4\") pod \"service-ca-58598695f7-fbkxp\" (UID: \"1491c729-3b1e-4f63-a65b-e64308d91c27\") " pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.340561 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.340476 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-fbkxp" Jul 22 18:57:27.538265 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.538234 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fbkxp"] Jul 22 18:57:27.540951 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:27.540930 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1491c729_3b1e_4f63_a65b_e64308d91c27.slice/crio-59b3ee609c1d14376c8283b1c55d875574c00e279643cd6cdc17a356de5c9090 WatchSource:0}: Error finding container 59b3ee609c1d14376c8283b1c55d875574c00e279643cd6cdc17a356de5c9090: Status 404 returned error can't find the container with id 59b3ee609c1d14376c8283b1c55d875574c00e279643cd6cdc17a356de5c9090 Jul 22 18:57:27.612441 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:27.612421 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mkx5n_8ae751a3-8d23-4553-ac23-3e752a6958b7/node-ca/0.log" Jul 22 18:57:28.013741 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.013709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x589_7578bff6-22d9-4bd1-981f-0fccb3b3bf8a/serve-healthcheck-canary/0.log" Jul 22 18:57:28.019824 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.019795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" event={"ID":"b45ec387-cde4-4b3e-9ad6-742c19bef34c","Type":"ContainerStarted","Data":"c66d2e9b1bc59d8a83927f747ad343ff2758f927d2ef8fd20d7f8c7adc0df19a"} Jul 22 18:57:28.020261 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.019879 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" event={"ID":"b45ec387-cde4-4b3e-9ad6-742c19bef34c","Type":"ContainerStarted","Data":"e54d4f91614bfbbbaf106ecc90b8aeaf690d87a732665a943edbedc2e18d0d9e"} Jul 22 18:57:28.020931 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.020910 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-fbkxp" event={"ID":"1491c729-3b1e-4f63-a65b-e64308d91c27","Type":"ContainerStarted","Data":"59b3ee609c1d14376c8283b1c55d875574c00e279643cd6cdc17a356de5c9090"} Jul 22 18:57:28.043314 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.043273 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-t4bjv" podStartSLOduration=1.934589406 podStartE2EDuration="4.043260168s" totalImagesPullingTime="2.108670762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:28.042657456 +0000 UTC m=+48.769218569" watchObservedRunningTime="2026-07-22 18:57:28.043260168 +0000 UTC m=+48.769821402" Jul 22 18:57:28.237903 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.237864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:28.238051 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:28.237973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:28.238051 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:28.238027 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:57:28.238179 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:28.238087 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:57:28.238179 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:28.238110 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:32.238088748 +0000 UTC m=+52.964649829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : secret "insights-runtime-extractor-tls" not found Jul 22 18:57:28.238179 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:28.238139 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:32.238123026 +0000 UTC m=+52.964684111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : configmap "kube-rbac-proxy" not found Jul 22 18:57:30.027108 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:30.027021 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-fbkxp" event={"ID":"1491c729-3b1e-4f63-a65b-e64308d91c27","Type":"ContainerStarted","Data":"787827f76e709fd7fef31e0514ec32abef8b9252aba51e8b2ee5d00731acb7c3"} Jul 22 18:57:30.044494 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:30.044445 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-fbkxp" podStartSLOduration=1.8605170229999999 podStartE2EDuration="4.044428884s" totalImagesPullingTime="2.183911861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:30.043529025 +0000 UTC m=+50.770090128" watchObservedRunningTime="2026-07-22 18:57:30.044428884 +0000 UTC m=+50.770989985" Jul 22 18:57:31.064409 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:31.064367 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:31.064853 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:31.064550 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:57:31.064853 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:31.064638 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls podName:a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb nodeName:}" failed. No retries permitted until 2026-07-22 18:57:47.064617406 +0000 UTC m=+67.791178500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls") pod "dns-default-dfkvn" (UID: "a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb") : secret "dns-default-metrics-tls" not found Jul 22 18:57:32.274864 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:32.274825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:32.275273 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:32.274888 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:32.275273 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:32.275001 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:57:32.275273 ip-10-0-137-176 kubenswrapper[2565]: E0722 18:57:32.275054 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls podName:6859a523-ead0-4896-bd3a-826f9998e504 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:40.275041216 +0000 UTC m=+61.001602295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls") pod "insights-runtime-extractor-flr6f" (UID: "6859a523-ead0-4896-bd3a-826f9998e504") : secret "insights-runtime-extractor-tls" not found Jul 22 18:57:32.275399 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:32.275368 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6859a523-ead0-4896-bd3a-826f9998e504-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:39.975841 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:39.975805 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9rsq" Jul 22 18:57:40.336560 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:40.336459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:40.338781 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:40.338758 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6859a523-ead0-4896-bd3a-826f9998e504-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-flr6f\" (UID: \"6859a523-ead0-4896-bd3a-826f9998e504\") " pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:40.371069 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:40.371040 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wspht\"" Jul 22 18:57:40.379136 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:40.379110 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-flr6f" Jul 22 18:57:40.523393 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:40.523363 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-flr6f"] Jul 22 18:57:40.526677 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:40.526647 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6859a523_ead0_4896_bd3a_826f9998e504.slice/crio-188892e1f653e4a15d131c2aea1b91dc4df62571916ba54fe8d3dc222838fef6 WatchSource:0}: Error finding container 188892e1f653e4a15d131c2aea1b91dc4df62571916ba54fe8d3dc222838fef6: Status 404 returned error can't find the container with id 188892e1f653e4a15d131c2aea1b91dc4df62571916ba54fe8d3dc222838fef6 Jul 22 18:57:41.060040 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:41.059947 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-flr6f" event={"ID":"6859a523-ead0-4896-bd3a-826f9998e504","Type":"ContainerStarted","Data":"188892e1f653e4a15d131c2aea1b91dc4df62571916ba54fe8d3dc222838fef6"} Jul 22 18:57:45.480221 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.480183 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:45.480221 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.480225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:45.482806 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.482784 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 18:57:45.482897 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.482817 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 18:57:45.493152 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.493122 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7f38ebc-93c7-4813-9b3f-3038502b046b-original-pull-secret\") pod \"global-pull-secret-syncer-5qpnv\" (UID: \"e7f38ebc-93c7-4813-9b3f-3038502b046b\") " pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:45.493282 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.493204 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e4e1837-2baa-4c5d-927b-70682f840b7a-metrics-certs\") pod \"network-metrics-daemon-vnnnq\" (UID: \"6e4e1837-2baa-4c5d-927b-70682f840b7a\") " pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:45.526029 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.525999 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5knzw\"" Jul 22 18:57:45.528000 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.527982 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qpnv" Jul 22 18:57:45.533886 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.533856 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnnnq" Jul 22 18:57:45.581366 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.581206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:45.584271 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.584243 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 18:57:45.594857 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.594586 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 18:57:45.604426 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.604376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7x65\" (UniqueName: \"kubernetes.io/projected/138b9c50-9adc-41a8-af93-38a9a81982c1-kube-api-access-f7x65\") pod \"network-check-target-5dwdc\" (UID: \"138b9c50-9adc-41a8-af93-38a9a81982c1\") " pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:45.659929 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.659898 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qpnv"] Jul 22 18:57:45.663077 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:45.663049 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f38ebc_93c7_4813_9b3f_3038502b046b.slice/crio-f47cd0a049a75d9bf1835b23aa579a9949576b4dadf6244ee2f5712b4cf2f007 WatchSource:0}: Error finding container f47cd0a049a75d9bf1835b23aa579a9949576b4dadf6244ee2f5712b4cf2f007: Status 404 returned error can't find the container with id f47cd0a049a75d9bf1835b23aa579a9949576b4dadf6244ee2f5712b4cf2f007 Jul 22 18:57:45.676437 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.676410 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vnnnq"] Jul 22 18:57:45.679197 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:45.679157 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4e1837_2baa_4c5d_927b_70682f840b7a.slice/crio-e88fdf2f0be4ea5b84c739baba51bf4d344390466404b6989fee9f19a28f8d9a WatchSource:0}: Error finding container e88fdf2f0be4ea5b84c739baba51bf4d344390466404b6989fee9f19a28f8d9a: Status 404 returned error can't find the container with id e88fdf2f0be4ea5b84c739baba51bf4d344390466404b6989fee9f19a28f8d9a Jul 22 18:57:45.820617 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.820574 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g8nmg\"" Jul 22 18:57:45.828599 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.828511 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:45.985595 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:45.985552 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5dwdc"] Jul 22 18:57:45.989679 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:45.989635 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138b9c50_9adc_41a8_af93_38a9a81982c1.slice/crio-2c81621f1f3dd2bc2809cb9b4b0ce611a26711ffbe954049e932bd3fca8953c8 WatchSource:0}: Error finding container 2c81621f1f3dd2bc2809cb9b4b0ce611a26711ffbe954049e932bd3fca8953c8: Status 404 returned error can't find the container with id 2c81621f1f3dd2bc2809cb9b4b0ce611a26711ffbe954049e932bd3fca8953c8 Jul 22 18:57:46.074861 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:46.074809 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qpnv" event={"ID":"e7f38ebc-93c7-4813-9b3f-3038502b046b","Type":"ContainerStarted","Data":"f47cd0a049a75d9bf1835b23aa579a9949576b4dadf6244ee2f5712b4cf2f007"} Jul 22 18:57:46.076929 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:46.076895 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5dwdc" event={"ID":"138b9c50-9adc-41a8-af93-38a9a81982c1","Type":"ContainerStarted","Data":"2c81621f1f3dd2bc2809cb9b4b0ce611a26711ffbe954049e932bd3fca8953c8"} Jul 22 18:57:46.078776 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:46.078692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-flr6f" event={"ID":"6859a523-ead0-4896-bd3a-826f9998e504","Type":"ContainerStarted","Data":"0a41919378476c93acca6907fdf6ba9cc804cafc594f909f45fc6ae420bbae1d"} Jul 22 18:57:46.080221 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:46.080194 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vnnnq" event={"ID":"6e4e1837-2baa-4c5d-927b-70682f840b7a","Type":"ContainerStarted","Data":"e88fdf2f0be4ea5b84c739baba51bf4d344390466404b6989fee9f19a28f8d9a"} Jul 22 18:57:47.093817 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:47.093775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:47.096827 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:47.096794 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb-metrics-tls\") pod \"dns-default-dfkvn\" (UID: \"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb\") " pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:47.290529 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:47.290485 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cc98d\"" Jul 22 18:57:47.298108 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:47.298077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:52.107905 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.107734 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5dwdc" event={"ID":"138b9c50-9adc-41a8-af93-38a9a81982c1","Type":"ContainerStarted","Data":"7d93dad5e8e5928ee22965e185725c169711dcdf3b179df9d33fab0d5a7752e0"} Jul 22 18:57:52.107905 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.107862 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:57:52.110549 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.110520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-flr6f" event={"ID":"6859a523-ead0-4896-bd3a-826f9998e504","Type":"ContainerStarted","Data":"2664f1f7c3588cd62657b70766892ba7bc23e595e7431011b2628f767d96772c"} Jul 22 18:57:52.113034 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.112980 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vnnnq" event={"ID":"6e4e1837-2baa-4c5d-927b-70682f840b7a","Type":"ContainerStarted","Data":"d3a50fb3808af1010d13e0ef8516b93330cb4fa611bde82e10f80656d2b922d8"} Jul 22 18:57:52.123855 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.123831 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dfkvn"] Jul 22 18:57:52.126384 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:52.126356 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a3c7ae_6b73_47b3_ac4f_2365fb5cdedb.slice/crio-2f12e31c9bb9593b7627a3c56647bd57d5df74cdc6c70b66fe5658db60ab031a WatchSource:0}: Error finding container 2f12e31c9bb9593b7627a3c56647bd57d5df74cdc6c70b66fe5658db60ab031a: Status 404 returned error can't find the container with id 2f12e31c9bb9593b7627a3c56647bd57d5df74cdc6c70b66fe5658db60ab031a Jul 22 18:57:52.129110 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.128575 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5dwdc" podStartSLOduration=67.158572376 podStartE2EDuration="1m13.128557632s" totalImagesPullingTime="5.969985256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:52.128224867 +0000 UTC m=+72.854785971" watchObservedRunningTime="2026-07-22 18:57:52.128557632 +0000 UTC m=+72.855118733" Jul 22 18:57:52.335710 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.335627 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl"] Jul 22 18:57:52.338067 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.338051 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5"] Jul 22 18:57:52.338220 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.338200 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.339970 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.339946 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.340899 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.340880 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 22 18:57:52.340899 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.340882 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 22 18:57:52.341047 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.340960 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 18:57:52.341047 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.340991 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 18:57:52.341232 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.341218 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 22 18:57:52.341384 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.341367 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 22 18:57:52.341689 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.341672 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 18:57:52.342532 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.342518 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 18:57:52.343493 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.343475 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zlbqr\"" Jul 22 18:57:52.348107 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.348079 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5"] Jul 22 18:57:52.355034 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.355010 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl"] Jul 22 18:57:52.437714 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437678 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.437714 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.437964 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf5b345a-babd-4af6-b27e-08e11c743314-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5\" (UID: \"bf5b345a-babd-4af6-b27e-08e11c743314\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.437964 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437758 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.437964 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jsg2\" (UniqueName: \"kubernetes.io/projected/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-kube-api-access-2jsg2\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.437964 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437812 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-ca\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.437964 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc2cp\" (UniqueName: \"kubernetes.io/projected/bf5b345a-babd-4af6-b27e-08e11c743314-kube-api-access-fc2cp\") pod \"managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5\" (UID: \"bf5b345a-babd-4af6-b27e-08e11c743314\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.437964 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.437878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-hub\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.538906 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.538874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jsg2\" (UniqueName: \"kubernetes.io/projected/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-kube-api-access-2jsg2\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.539109 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.538948 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-ca\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.539109 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.538984 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc2cp\" (UniqueName: \"kubernetes.io/projected/bf5b345a-babd-4af6-b27e-08e11c743314-kube-api-access-fc2cp\") pod \"managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5\" (UID: \"bf5b345a-babd-4af6-b27e-08e11c743314\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.539109 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.539005 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-hub\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.539109 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.539044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.539109 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.539065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.539109 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.539089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf5b345a-babd-4af6-b27e-08e11c743314-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5\" (UID: \"bf5b345a-babd-4af6-b27e-08e11c743314\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.539416 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.539112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.540016 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.539990 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.541995 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.541969 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.542117 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.542097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-ca\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.542251 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.542226 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.542375 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.542296 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf5b345a-babd-4af6-b27e-08e11c743314-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5\" (UID: \"bf5b345a-babd-4af6-b27e-08e11c743314\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.542498 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.542483 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-hub\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.552285 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.552260 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc2cp\" (UniqueName: \"kubernetes.io/projected/bf5b345a-babd-4af6-b27e-08e11c743314-kube-api-access-fc2cp\") pod \"managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5\" (UID: \"bf5b345a-babd-4af6-b27e-08e11c743314\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.555669 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.555596 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jsg2\" (UniqueName: \"kubernetes.io/projected/be0d89be-3cd8-4834-90e1-eb8e6d4b5873-kube-api-access-2jsg2\") pod \"cluster-proxy-proxy-agent-78465b89bc-wq6nl\" (UID: \"be0d89be-3cd8-4834-90e1-eb8e6d4b5873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.662768 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.662709 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" Jul 22 18:57:52.667936 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.667912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" Jul 22 18:57:52.834128 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.834097 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl"] Jul 22 18:57:52.838944 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:52.838554 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe0d89be_3cd8_4834_90e1_eb8e6d4b5873.slice/crio-0177dd8ea17e1e51fe3e47e80855b3258e38b663e2c53d529778c8a1ae4c11bd WatchSource:0}: Error finding container 0177dd8ea17e1e51fe3e47e80855b3258e38b663e2c53d529778c8a1ae4c11bd: Status 404 returned error can't find the container with id 0177dd8ea17e1e51fe3e47e80855b3258e38b663e2c53d529778c8a1ae4c11bd Jul 22 18:57:52.853361 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:52.853298 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5"] Jul 22 18:57:52.857427 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:57:52.857384 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf5b345a_babd_4af6_b27e_08e11c743314.slice/crio-c3bfd75bc51bdf2fd56cf4b20946004778860fe819a349dbe2f450f05ede52d6 WatchSource:0}: Error finding container c3bfd75bc51bdf2fd56cf4b20946004778860fe819a349dbe2f450f05ede52d6: Status 404 returned error can't find the container with id c3bfd75bc51bdf2fd56cf4b20946004778860fe819a349dbe2f450f05ede52d6 Jul 22 18:57:53.121945 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.121855 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vnnnq" event={"ID":"6e4e1837-2baa-4c5d-927b-70682f840b7a","Type":"ContainerStarted","Data":"4b228771adc1e272aa72d9d7f3fd2966dc9453d3b0c10071005ee02a97c292e0"} Jul 22 18:57:53.123473 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.123442 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" event={"ID":"be0d89be-3cd8-4834-90e1-eb8e6d4b5873","Type":"ContainerStarted","Data":"0177dd8ea17e1e51fe3e47e80855b3258e38b663e2c53d529778c8a1ae4c11bd"} Jul 22 18:57:53.125011 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.124951 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dfkvn" event={"ID":"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb","Type":"ContainerStarted","Data":"2f12e31c9bb9593b7627a3c56647bd57d5df74cdc6c70b66fe5658db60ab031a"} Jul 22 18:57:53.127427 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.127367 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qpnv" event={"ID":"e7f38ebc-93c7-4813-9b3f-3038502b046b","Type":"ContainerStarted","Data":"7f992da4ad93608ea42fa32328f916d46449c3f833eb809793084f9fad0c3c57"} Jul 22 18:57:53.128862 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.128802 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" event={"ID":"bf5b345a-babd-4af6-b27e-08e11c743314","Type":"ContainerStarted","Data":"c3bfd75bc51bdf2fd56cf4b20946004778860fe819a349dbe2f450f05ede52d6"} Jul 22 18:57:53.139674 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.139582 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vnnnq" podStartSLOduration=69.245210492 podStartE2EDuration="1m14.139560674s" totalImagesPullingTime="4.894350182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:53.138406112 +0000 UTC m=+73.864967215" watchObservedRunningTime="2026-07-22 18:57:53.139560674 +0000 UTC m=+73.866121776" Jul 22 18:57:53.157527 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:53.157465 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5qpnv" podStartSLOduration=67.840937777 podStartE2EDuration="1m14.157441849s" totalImagesPullingTime="6.316504072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:53.155598752 +0000 UTC m=+73.882159856" watchObservedRunningTime="2026-07-22 18:57:53.157441849 +0000 UTC m=+73.884002949" Jul 22 18:57:54.137206 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:54.136696 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dfkvn" event={"ID":"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb","Type":"ContainerStarted","Data":"7c6db8f0aac296fbc868ad3bd696ad9564c08dd080124b4aeb7df8c72a2a879c"} Jul 22 18:57:54.137206 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:54.136774 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dfkvn" event={"ID":"a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb","Type":"ContainerStarted","Data":"9b0bcc9fb9ce77a17305006fac95019f0706c5f9278548ef1ad3301717079d59"} Jul 22 18:57:54.137206 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:54.137081 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dfkvn" Jul 22 18:57:59.159311 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.159273 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-flr6f" event={"ID":"6859a523-ead0-4896-bd3a-826f9998e504","Type":"ContainerStarted","Data":"7996f852e75070f6bec26723eb26e3c1dde819b640691d3ea2de3dc98f166637"} Jul 22 18:57:59.161318 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.161289 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" event={"ID":"be0d89be-3cd8-4834-90e1-eb8e6d4b5873","Type":"ContainerStarted","Data":"9604ed093d65c7a6ae71fb2ae4267c2dee10774e11bc9a0fe2c4dfef698b0a22"} Jul 22 18:57:59.161400 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.161341 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" event={"ID":"be0d89be-3cd8-4834-90e1-eb8e6d4b5873","Type":"ContainerStarted","Data":"f67c6a46a41e963562de695dee4b57cb0dcb90293e58ffab8f18d5b09bed0ce7"} Jul 22 18:57:59.161400 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.161362 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" event={"ID":"be0d89be-3cd8-4834-90e1-eb8e6d4b5873","Type":"ContainerStarted","Data":"55a7c267a7222607cdab364121621b8a9365f139bf72251cefb9e31c94061af5"} Jul 22 18:57:59.162685 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.162661 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" event={"ID":"bf5b345a-babd-4af6-b27e-08e11c743314","Type":"ContainerStarted","Data":"04bc3b3a6f5a534d180c332c24ca1cb10ff1ef4f0d2bf01ab5e8f02620dbfa08"} Jul 22 18:57:59.177403 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.177352 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dfkvn" podStartSLOduration=42.502091721 podStartE2EDuration="44.177334227s" totalImagesPullingTime="1.675242506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:54.15826078 +0000 UTC m=+74.884821884" watchObservedRunningTime="2026-07-22 18:57:59.177334227 +0000 UTC m=+79.903895330" Jul 22 18:57:59.177580 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.177551 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-flr6f" podStartSLOduration=17.204304868 podStartE2EDuration="35.177537231s" totalImagesPullingTime="17.973232363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:57:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:59.176894561 +0000 UTC m=+79.903455659" watchObservedRunningTime="2026-07-22 18:57:59.177537231 +0000 UTC m=+79.904098346" Jul 22 18:57:59.195350 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.195301 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78465b89bc-wq6nl" podStartSLOduration=1.385236584 podStartE2EDuration="7.195286565s" totalImagesPullingTime="5.810049981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:59.194863669 +0000 UTC m=+79.921424774" watchObservedRunningTime="2026-07-22 18:57:59.195286565 +0000 UTC m=+79.921847667" Jul 22 18:57:59.212565 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:57:59.212516 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ff6bfcfbb-l65b5" podStartSLOduration=1.431301631 podStartE2EDuration="7.21250359s" totalImagesPullingTime="5.781201959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:59.210527752 +0000 UTC m=+79.937088853" watchObservedRunningTime="2026-07-22 18:57:59.21250359 +0000 UTC m=+79.939064691" Jul 22 18:58:05.148221 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:05.148084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dfkvn" Jul 22 18:58:07.542487 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.542456 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-czvpw"] Jul 22 18:58:07.547152 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.547127 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.549411 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.549384 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fvp2b\"" Jul 22 18:58:07.549707 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.549686 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 18:58:07.550446 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.550426 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 18:58:07.550563 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.550529 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 18:58:07.550951 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.550928 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 18:58:07.551057 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.551038 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 18:58:07.551186 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.551167 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 18:58:07.658236 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzht9\" (UniqueName: \"kubernetes.io/projected/dc485f2a-051f-4af7-87c7-36296e7430b1-kube-api-access-xzht9\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658236 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658241 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc485f2a-051f-4af7-87c7-36296e7430b1-metrics-client-ca\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658507 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658268 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-tls\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658507 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658507 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658417 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-root\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658507 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-textfile\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658744 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658744 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658594 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-sys\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.658744 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.658640 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-wtmp\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759294 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759469 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759307 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-sys\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759469 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759360 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-sys\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759469 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759392 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-wtmp\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759469 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759419 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzht9\" (UniqueName: \"kubernetes.io/projected/dc485f2a-051f-4af7-87c7-36296e7430b1-kube-api-access-xzht9\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759469 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759438 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc485f2a-051f-4af7-87c7-36296e7430b1-metrics-client-ca\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759469 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759458 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-tls\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759481 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759517 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-root\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759539 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-textfile\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759631 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-wtmp\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759775 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759702 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc485f2a-051f-4af7-87c7-36296e7430b1-root\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.759944 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759853 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-textfile\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.760004 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.759943 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.760537 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.760504 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc485f2a-051f-4af7-87c7-36296e7430b1-metrics-client-ca\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.765184 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.762029 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.767241 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.767222 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc485f2a-051f-4af7-87c7-36296e7430b1-node-exporter-tls\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.770475 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.770445 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzht9\" (UniqueName: \"kubernetes.io/projected/dc485f2a-051f-4af7-87c7-36296e7430b1-kube-api-access-xzht9\") pod \"node-exporter-czvpw\" (UID: \"dc485f2a-051f-4af7-87c7-36296e7430b1\") " pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.857575 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:07.857486 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-czvpw" Jul 22 18:58:07.869916 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:58:07.869884 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc485f2a_051f_4af7_87c7_36296e7430b1.slice/crio-13ebffc5a594afa65c25da635d030af1a40905a804ce742122ec4c9a4a646c12 WatchSource:0}: Error finding container 13ebffc5a594afa65c25da635d030af1a40905a804ce742122ec4c9a4a646c12: Status 404 returned error can't find the container with id 13ebffc5a594afa65c25da635d030af1a40905a804ce742122ec4c9a4a646c12 Jul 22 18:58:08.193670 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:08.193636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czvpw" event={"ID":"dc485f2a-051f-4af7-87c7-36296e7430b1","Type":"ContainerStarted","Data":"13ebffc5a594afa65c25da635d030af1a40905a804ce742122ec4c9a4a646c12"} Jul 22 18:58:10.200893 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:10.200855 2565 generic.go:356] "Generic (PLEG): container finished" podID="dc485f2a-051f-4af7-87c7-36296e7430b1" containerID="16604d49ae776d1e25e85b04e5076cc7bd445028aac054b988c6087e108cdee0" exitCode=0 Jul 22 18:58:10.201395 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:10.200921 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czvpw" event={"ID":"dc485f2a-051f-4af7-87c7-36296e7430b1","Type":"ContainerDied","Data":"16604d49ae776d1e25e85b04e5076cc7bd445028aac054b988c6087e108cdee0"} Jul 22 18:58:11.206766 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.206724 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czvpw" event={"ID":"dc485f2a-051f-4af7-87c7-36296e7430b1","Type":"ContainerStarted","Data":"b562bcd56c460473c3bef3a36ccd70e938afff976e5ac674558ad9bb85fabef4"} Jul 22 18:58:11.207139 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.206823 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czvpw" event={"ID":"dc485f2a-051f-4af7-87c7-36296e7430b1","Type":"ContainerStarted","Data":"d39a2332dedbddac96292ccdc5b684b094cf6548db28bf471cf658fd7701ec69"} Jul 22 18:58:11.230501 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.230446 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-czvpw" podStartSLOduration=2.878864982 podStartE2EDuration="4.230429341s" totalImagesPullingTime="1.288029642s" totalInitContainerRuntime="63.534717ms" isStatefulPod=true podCreationTimestamp="2026-07-22 18:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:58:11.228750553 +0000 UTC m=+91.955311657" watchObservedRunningTime="2026-07-22 18:58:11.230429341 +0000 UTC m=+91.956990443" Jul 22 18:58:11.906870 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.906837 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-798d7cd4d9-xfzv7"] Jul 22 18:58:11.910041 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.910026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.912392 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.912342 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 22 18:58:11.913538 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.913508 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5gw8w\"" Jul 22 18:58:11.913661 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.913583 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 22 18:58:11.913661 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.913641 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 22 18:58:11.913766 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.913678 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bl6mj4hv8kmij\"" Jul 22 18:58:11.913849 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.913835 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 18:58:11.919734 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.919714 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-798d7cd4d9-xfzv7"] Jul 22 18:58:11.990811 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.990772 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-client-ca-bundle\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.990811 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.990808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67ld4\" (UniqueName: \"kubernetes.io/projected/a5466ae1-d2b9-4842-9145-b50c3223b816-kube-api-access-67ld4\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.991037 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.990829 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a5466ae1-d2b9-4842-9145-b50c3223b816-metrics-server-audit-profiles\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.991037 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.990851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-secret-metrics-server-tls\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.991037 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.990946 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5466ae1-d2b9-4842-9145-b50c3223b816-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.991037 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.990977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a5466ae1-d2b9-4842-9145-b50c3223b816-audit-log\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:11.991037 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:11.991001 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-secret-metrics-server-client-certs\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091729 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-client-ca-bundle\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091729 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67ld4\" (UniqueName: \"kubernetes.io/projected/a5466ae1-d2b9-4842-9145-b50c3223b816-kube-api-access-67ld4\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091957 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a5466ae1-d2b9-4842-9145-b50c3223b816-metrics-server-audit-profiles\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091957 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-secret-metrics-server-tls\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091957 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5466ae1-d2b9-4842-9145-b50c3223b816-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091957 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091870 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a5466ae1-d2b9-4842-9145-b50c3223b816-audit-log\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.091957 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.091903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-secret-metrics-server-client-certs\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.092336 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.092307 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a5466ae1-d2b9-4842-9145-b50c3223b816-audit-log\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.092625 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.092584 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5466ae1-d2b9-4842-9145-b50c3223b816-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.092881 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.092863 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a5466ae1-d2b9-4842-9145-b50c3223b816-metrics-server-audit-profiles\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.094156 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.094126 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-client-ca-bundle\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.094294 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.094275 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-secret-metrics-server-client-certs\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.094294 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.094289 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a5466ae1-d2b9-4842-9145-b50c3223b816-secret-metrics-server-tls\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.099809 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.099787 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67ld4\" (UniqueName: \"kubernetes.io/projected/a5466ae1-d2b9-4842-9145-b50c3223b816-kube-api-access-67ld4\") pod \"metrics-server-798d7cd4d9-xfzv7\" (UID: \"a5466ae1-d2b9-4842-9145-b50c3223b816\") " pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.219151 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.219120 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:12.353427 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:12.353397 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-798d7cd4d9-xfzv7"] Jul 22 18:58:12.356588 ip-10-0-137-176 kubenswrapper[2565]: W0722 18:58:12.356551 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5466ae1_d2b9_4842_9145_b50c3223b816.slice/crio-04d6b800e4e6911da464bb218e588cb864c4d00253e76a9ed4e63e585b282a63 WatchSource:0}: Error finding container 04d6b800e4e6911da464bb218e588cb864c4d00253e76a9ed4e63e585b282a63: Status 404 returned error can't find the container with id 04d6b800e4e6911da464bb218e588cb864c4d00253e76a9ed4e63e585b282a63 Jul 22 18:58:13.214933 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:13.214892 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" event={"ID":"a5466ae1-d2b9-4842-9145-b50c3223b816","Type":"ContainerStarted","Data":"04d6b800e4e6911da464bb218e588cb864c4d00253e76a9ed4e63e585b282a63"} Jul 22 18:58:14.219504 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:14.219470 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" event={"ID":"a5466ae1-d2b9-4842-9145-b50c3223b816","Type":"ContainerStarted","Data":"ab0af373c68a878b02feee5c86c26e5ebc2dde95cfbb25e63817e9d40b74307f"} Jul 22 18:58:23.131162 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:23.131129 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5dwdc" Jul 22 18:58:23.149070 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:23.149020 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" podStartSLOduration=10.875775407999999 podStartE2EDuration="12.149003338s" totalImagesPullingTime="1.27322793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:58:14.237161842 +0000 UTC m=+94.963722945" watchObservedRunningTime="2026-07-22 18:58:23.149003338 +0000 UTC m=+103.875564440" Jul 22 18:58:32.219952 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:32.219914 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:32.220338 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:32.219983 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:52.226104 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:52.226070 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 18:58:52.232160 ip-10-0-137-176 kubenswrapper[2565]: I0722 18:58:52.232129 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-798d7cd4d9-xfzv7" Jul 22 19:01:18.867855 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.867817 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jrhrq"] Jul 22 19:01:18.871958 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.871932 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:18.874428 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.874408 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 19:01:18.875388 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.875370 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-m6cmt\"" Jul 22 19:01:18.875490 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.875410 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 19:01:18.878210 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.878163 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jrhrq"] Jul 22 19:01:18.978778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.978742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012ef609-8b88-4374-8743-3b528e19e0b7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jrhrq\" (UID: \"012ef609-8b88-4374-8743-3b528e19e0b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:18.978963 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:18.978787 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lsbk\" (UniqueName: \"kubernetes.io/projected/012ef609-8b88-4374-8743-3b528e19e0b7-kube-api-access-5lsbk\") pod \"cert-manager-webhook-6486ffd997-jrhrq\" (UID: \"012ef609-8b88-4374-8743-3b528e19e0b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:19.080007 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.079960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012ef609-8b88-4374-8743-3b528e19e0b7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jrhrq\" (UID: \"012ef609-8b88-4374-8743-3b528e19e0b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:19.080156 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.080021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lsbk\" (UniqueName: \"kubernetes.io/projected/012ef609-8b88-4374-8743-3b528e19e0b7-kube-api-access-5lsbk\") pod \"cert-manager-webhook-6486ffd997-jrhrq\" (UID: \"012ef609-8b88-4374-8743-3b528e19e0b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:19.088602 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.088568 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012ef609-8b88-4374-8743-3b528e19e0b7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jrhrq\" (UID: \"012ef609-8b88-4374-8743-3b528e19e0b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:19.088772 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.088654 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lsbk\" (UniqueName: \"kubernetes.io/projected/012ef609-8b88-4374-8743-3b528e19e0b7-kube-api-access-5lsbk\") pod \"cert-manager-webhook-6486ffd997-jrhrq\" (UID: \"012ef609-8b88-4374-8743-3b528e19e0b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:19.181027 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.180982 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:19.308023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.307997 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jrhrq"] Jul 22 19:01:19.310392 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:01:19.310361 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod012ef609_8b88_4374_8743_3b528e19e0b7.slice/crio-f67ea7916754d9493dd4e3d71c915edf84e2efb0f2b5fc2fb9bb7b77f507ef95 WatchSource:0}: Error finding container f67ea7916754d9493dd4e3d71c915edf84e2efb0f2b5fc2fb9bb7b77f507ef95: Status 404 returned error can't find the container with id f67ea7916754d9493dd4e3d71c915edf84e2efb0f2b5fc2fb9bb7b77f507ef95 Jul 22 19:01:19.842365 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:19.842329 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" event={"ID":"012ef609-8b88-4374-8743-3b528e19e0b7","Type":"ContainerStarted","Data":"f67ea7916754d9493dd4e3d71c915edf84e2efb0f2b5fc2fb9bb7b77f507ef95"} Jul 22 19:01:22.383620 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.383584 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vvljr"] Jul 22 19:01:22.386545 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.386527 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.390376 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.390357 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lpkgw\"" Jul 22 19:01:22.415973 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.415945 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vvljr"] Jul 22 19:01:22.509740 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.509700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb-kube-api-access-6g2lc\") pod \"cert-manager-55b4d954d5-vvljr\" (UID: \"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb\") " pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.509902 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.509748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-vvljr\" (UID: \"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb\") " pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.610107 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.610026 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb-kube-api-access-6g2lc\") pod \"cert-manager-55b4d954d5-vvljr\" (UID: \"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb\") " pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.610107 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.610067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-vvljr\" (UID: \"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb\") " pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.629027 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.628993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb-kube-api-access-6g2lc\") pod \"cert-manager-55b4d954d5-vvljr\" (UID: \"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb\") " pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.630186 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.630169 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-vvljr\" (UID: \"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb\") " pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.695618 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.695574 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vvljr" Jul 22 19:01:22.829422 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.829393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vvljr"] Jul 22 19:01:22.832156 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:01:22.832132 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf6ef244_a2c2_4b9f_bbda_dd6a4465f1eb.slice/crio-3759944f4c04acd9cda204e540da137b16fc589870bd14c98b8bc0543af80b93 WatchSource:0}: Error finding container 3759944f4c04acd9cda204e540da137b16fc589870bd14c98b8bc0543af80b93: Status 404 returned error can't find the container with id 3759944f4c04acd9cda204e540da137b16fc589870bd14c98b8bc0543af80b93 Jul 22 19:01:22.854557 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.854528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" event={"ID":"012ef609-8b88-4374-8743-3b528e19e0b7","Type":"ContainerStarted","Data":"2e936cae46b1c936cdec07e789baba98b175c694f4a6ca3f6252b1887094b0c3"} Jul 22 19:01:22.854662 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.854641 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:22.855578 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.855561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vvljr" event={"ID":"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb","Type":"ContainerStarted","Data":"3759944f4c04acd9cda204e540da137b16fc589870bd14c98b8bc0543af80b93"} Jul 22 19:01:22.877051 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:22.877006 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" podStartSLOduration=1.915360419 podStartE2EDuration="4.876990042s" totalImagesPullingTime="2.961629623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:22.875934703 +0000 UTC m=+283.602495807" watchObservedRunningTime="2026-07-22 19:01:22.876990042 +0000 UTC m=+283.603551143" Jul 22 19:01:23.860902 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:23.860864 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vvljr" event={"ID":"bf6ef244-a2c2-4b9f-bbda-dd6a4465f1eb","Type":"ContainerStarted","Data":"5bcafc781c5ac61e7df11b1442f6bab6692b5f09b845cb40a70a24245a7ad983"} Jul 22 19:01:23.907118 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:23.907060 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-vvljr" podStartSLOduration=1.9070397479999999 podStartE2EDuration="1.907039748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:23.90655351 +0000 UTC m=+284.633114612" watchObservedRunningTime="2026-07-22 19:01:23.907039748 +0000 UTC m=+284.633600850" Jul 22 19:01:28.863587 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:28.863556 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jrhrq" Jul 22 19:01:39.688173 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:39.688142 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 19:01:46.141092 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.141058 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh"] Jul 22 19:01:46.144126 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.144110 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.147439 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.147413 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 19:01:46.147629 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.147480 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 19:01:46.147629 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.147503 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-s4dgc\"" Jul 22 19:01:46.147629 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.147532 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 19:01:46.147629 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.147481 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 19:01:46.147928 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.147914 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 19:01:46.158084 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.158061 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh"] Jul 22 19:01:46.163840 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.163820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3319b9d1-894f-4f36-b1e9-cde27f62b721-manager-config\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.163944 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.163866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3319b9d1-894f-4f36-b1e9-cde27f62b721-cert\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.163944 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.163892 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwdv6\" (UniqueName: \"kubernetes.io/projected/3319b9d1-894f-4f36-b1e9-cde27f62b721-kube-api-access-hwdv6\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.164020 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.163947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3319b9d1-894f-4f36-b1e9-cde27f62b721-metrics-cert\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.264987 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.264957 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3319b9d1-894f-4f36-b1e9-cde27f62b721-cert\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.265195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.264999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwdv6\" (UniqueName: \"kubernetes.io/projected/3319b9d1-894f-4f36-b1e9-cde27f62b721-kube-api-access-hwdv6\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.265195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.265023 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3319b9d1-894f-4f36-b1e9-cde27f62b721-metrics-cert\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.265195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.265089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3319b9d1-894f-4f36-b1e9-cde27f62b721-manager-config\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.265819 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.265798 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3319b9d1-894f-4f36-b1e9-cde27f62b721-manager-config\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.267405 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.267380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3319b9d1-894f-4f36-b1e9-cde27f62b721-metrics-cert\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.267546 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.267525 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3319b9d1-894f-4f36-b1e9-cde27f62b721-cert\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.274003 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.273982 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwdv6\" (UniqueName: \"kubernetes.io/projected/3319b9d1-894f-4f36-b1e9-cde27f62b721-kube-api-access-hwdv6\") pod \"lws-controller-manager-7dd84c44d8-p2jnh\" (UID: \"3319b9d1-894f-4f36-b1e9-cde27f62b721\") " pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.452784 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.452752 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:46.591339 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.591315 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh"] Jul 22 19:01:46.593896 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:01:46.593869 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3319b9d1_894f_4f36_b1e9_cde27f62b721.slice/crio-41101dfb5e23d3186ecd19a6347037a0f7ed6853a089ead71b558ee1781608be WatchSource:0}: Error finding container 41101dfb5e23d3186ecd19a6347037a0f7ed6853a089ead71b558ee1781608be: Status 404 returned error can't find the container with id 41101dfb5e23d3186ecd19a6347037a0f7ed6853a089ead71b558ee1781608be Jul 22 19:01:46.595818 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.595802 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:01:46.936332 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:46.936298 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" event={"ID":"3319b9d1-894f-4f36-b1e9-cde27f62b721","Type":"ContainerStarted","Data":"41101dfb5e23d3186ecd19a6347037a0f7ed6853a089ead71b558ee1781608be"} Jul 22 19:01:50.868554 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.868518 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66"] Jul 22 19:01:50.871293 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.871276 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:50.881163 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.881141 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 19:01:50.881846 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.881831 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 19:01:50.881895 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.881833 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 19:01:50.882144 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.882132 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lzcck\"" Jul 22 19:01:50.882373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.882360 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 19:01:50.895177 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.895149 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66"] Jul 22 19:01:50.900251 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.900222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:50.900251 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.900256 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-manifests\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:50.900476 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.900283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-tmp\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:50.900476 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.900377 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j2zh\" (UniqueName: \"kubernetes.io/projected/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-kube-api-access-4j2zh\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:50.900476 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.900431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-webhook-cert\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:50.953421 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.953388 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" event={"ID":"3319b9d1-894f-4f36-b1e9-cde27f62b721","Type":"ContainerStarted","Data":"9188627e999c3a194360ce20bf3ba2ccde7dd97f75f3a67eaa1473c1db1782bb"} Jul 22 19:01:50.953590 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.953518 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:01:50.974149 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:50.974098 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" podStartSLOduration=1.499019206 podStartE2EDuration="4.974080239s" totalImagesPullingTime="3.475061033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:50.973780328 +0000 UTC m=+311.700341429" watchObservedRunningTime="2026-07-22 19:01:50.974080239 +0000 UTC m=+311.700641341" Jul 22 19:01:51.001398 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.001360 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j2zh\" (UniqueName: \"kubernetes.io/projected/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-kube-api-access-4j2zh\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.001594 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.001406 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-webhook-cert\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.001594 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.001473 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.001594 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.001491 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-manifests\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.001594 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.001507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-tmp\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.002220 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.002195 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-manifests\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.003943 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.003917 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-tmp\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.004047 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.004004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.004160 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.004142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-webhook-cert\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.009446 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.009422 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j2zh\" (UniqueName: \"kubernetes.io/projected/37dd099c-c1ea-4c86-9bf7-e927cffbb93e-kube-api-access-4j2zh\") pod \"opendatahub-operator-controller-manager-cb586fcb5-qtv66\" (UID: \"37dd099c-c1ea-4c86-9bf7-e927cffbb93e\") " pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.180736 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.180693 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:51.330526 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.330493 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66"] Jul 22 19:01:51.333583 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:01:51.333555 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37dd099c_c1ea_4c86_9bf7_e927cffbb93e.slice/crio-2ab92ffbb137d085b359294f32d7a320becc5c0dc6c4026bfb23df7c62b0609d WatchSource:0}: Error finding container 2ab92ffbb137d085b359294f32d7a320becc5c0dc6c4026bfb23df7c62b0609d: Status 404 returned error can't find the container with id 2ab92ffbb137d085b359294f32d7a320becc5c0dc6c4026bfb23df7c62b0609d Jul 22 19:01:51.960318 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:51.960277 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" event={"ID":"37dd099c-c1ea-4c86-9bf7-e927cffbb93e","Type":"ContainerStarted","Data":"2ab92ffbb137d085b359294f32d7a320becc5c0dc6c4026bfb23df7c62b0609d"} Jul 22 19:01:54.974753 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:54.974718 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" event={"ID":"37dd099c-c1ea-4c86-9bf7-e927cffbb93e","Type":"ContainerStarted","Data":"30a6344171072dcced1155183c81ca9d9e92b5c0ba23ce61133f47f39ddd3480"} Jul 22 19:01:55.979958 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:55.979925 2565 generic.go:356] "Generic (PLEG): container finished" podID="37dd099c-c1ea-4c86-9bf7-e927cffbb93e" containerID="30a6344171072dcced1155183c81ca9d9e92b5c0ba23ce61133f47f39ddd3480" exitCode=0 Jul 22 19:01:55.980331 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:55.979994 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" event={"ID":"37dd099c-c1ea-4c86-9bf7-e927cffbb93e","Type":"ContainerDied","Data":"30a6344171072dcced1155183c81ca9d9e92b5c0ba23ce61133f47f39ddd3480"} Jul 22 19:01:57.989367 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:57.989325 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" event={"ID":"37dd099c-c1ea-4c86-9bf7-e927cffbb93e","Type":"ContainerStarted","Data":"eca456ffa7c23faf6bc0e1eeceeb29e2207f57c1491bc90a5ef644b5d2d77d41"} Jul 22 19:01:57.989779 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:57.989461 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:01:58.014096 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:01:58.014044 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" podStartSLOduration=2.344688113 podStartE2EDuration="8.014024228s" totalImagesPullingTime="4.597026463s" totalInitContainerRuntime="1.072309652s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:58.013033309 +0000 UTC m=+318.739594444" watchObservedRunningTime="2026-07-22 19:01:58.014024228 +0000 UTC m=+318.740585331" Jul 22 19:02:01.963542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:01.963509 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dd84c44d8-p2jnh" Jul 22 19:02:08.995135 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:08.995106 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-cb586fcb5-qtv66" Jul 22 19:02:30.464228 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.464195 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx"] Jul 22 19:02:30.472887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.472867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.476247 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.476221 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 19:02:30.476382 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.476365 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 19:02:30.476696 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.476466 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pr4db\"" Jul 22 19:02:30.477454 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.477431 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 19:02:30.483163 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.483138 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx"] Jul 22 19:02:30.620458 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620424 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620458 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620466 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620488 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9hcw\" (UniqueName: \"kubernetes.io/projected/a01c936f-1571-44fd-aad9-cede33668e20-kube-api-access-p9hcw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01c936f-1571-44fd-aad9-cede33668e20-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01c936f-1571-44fd-aad9-cede33668e20-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620584 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01c936f-1571-44fd-aad9-cede33668e20-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620636 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620672 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.620751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.620697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.721910 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.721828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.721910 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.721868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9hcw\" (UniqueName: \"kubernetes.io/projected/a01c936f-1571-44fd-aad9-cede33668e20-kube-api-access-p9hcw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.721910 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.721910 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01c936f-1571-44fd-aad9-cede33668e20-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722164 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722084 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01c936f-1571-44fd-aad9-cede33668e20-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722164 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01c936f-1571-44fd-aad9-cede33668e20-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722271 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722192 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722271 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722359 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722277 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722359 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722297 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.722803 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.722318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.724657 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.724602 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01c936f-1571-44fd-aad9-cede33668e20-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.724783 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.724764 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.727719 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.724856 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.727719 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.725408 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.727719 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.725495 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01c936f-1571-44fd-aad9-cede33668e20-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.728137 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.728113 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01c936f-1571-44fd-aad9-cede33668e20-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.744623 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.744582 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01c936f-1571-44fd-aad9-cede33668e20-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.748534 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.748501 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9hcw\" (UniqueName: \"kubernetes.io/projected/a01c936f-1571-44fd-aad9-cede33668e20-kube-api-access-p9hcw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nq7wx\" (UID: \"a01c936f-1571-44fd-aad9-cede33668e20\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.784373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.784337 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:30.923575 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:30.923549 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx"] Jul 22 19:02:30.924892 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:02:30.924871 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01c936f_1571_44fd_aad9_cede33668e20.slice/crio-96dfd58ddee772584fda8cf803b4d7ccad8c4c244ba80cdaef2e3e0ff68ba7df WatchSource:0}: Error finding container 96dfd58ddee772584fda8cf803b4d7ccad8c4c244ba80cdaef2e3e0ff68ba7df: Status 404 returned error can't find the container with id 96dfd58ddee772584fda8cf803b4d7ccad8c4c244ba80cdaef2e3e0ff68ba7df Jul 22 19:02:31.103218 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:31.103100 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" event={"ID":"a01c936f-1571-44fd-aad9-cede33668e20","Type":"ContainerStarted","Data":"96dfd58ddee772584fda8cf803b4d7ccad8c4c244ba80cdaef2e3e0ff68ba7df"} Jul 22 19:02:34.670297 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:34.670251 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:02:34.670578 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:34.670363 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:02:34.670578 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:34.670393 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:02:35.120476 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:35.120375 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" event={"ID":"a01c936f-1571-44fd-aad9-cede33668e20","Type":"ContainerStarted","Data":"6a4c74a1a15670b0ab38ecea4594a80c4ba7603a4d1d8c12df29c52619439cb1"} Jul 22 19:02:35.147041 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:35.146992 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" podStartSLOduration=1.403822101 podStartE2EDuration="5.146976251s" totalImagesPullingTime="3.74315415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:02:35.145364231 +0000 UTC m=+355.871925334" watchObservedRunningTime="2026-07-22 19:02:35.146976251 +0000 UTC m=+355.873537353" Jul 22 19:02:35.785083 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:35.785047 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:35.789332 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:35.789310 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:36.124534 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:36.124451 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:02:36.125472 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:02:36.125453 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nq7wx" Jul 22 19:03:12.395578 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.395545 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lv6mz"] Jul 22 19:03:12.405505 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.405479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:12.407966 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.407938 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 19:03:12.408114 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.408094 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nh9hp\"" Jul 22 19:03:12.408371 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.408353 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 19:03:12.415709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.415688 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lv6mz"] Jul 22 19:03:12.560707 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.560661 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqfsb\" (UniqueName: \"kubernetes.io/projected/ca948a35-f134-4cb8-8906-a517d433a52b-kube-api-access-fqfsb\") pod \"kuadrant-operator-catalog-lv6mz\" (UID: \"ca948a35-f134-4cb8-8906-a517d433a52b\") " pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:12.661697 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.661668 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfsb\" (UniqueName: \"kubernetes.io/projected/ca948a35-f134-4cb8-8906-a517d433a52b-kube-api-access-fqfsb\") pod \"kuadrant-operator-catalog-lv6mz\" (UID: \"ca948a35-f134-4cb8-8906-a517d433a52b\") " pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:12.670063 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.670037 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfsb\" (UniqueName: \"kubernetes.io/projected/ca948a35-f134-4cb8-8906-a517d433a52b-kube-api-access-fqfsb\") pod \"kuadrant-operator-catalog-lv6mz\" (UID: \"ca948a35-f134-4cb8-8906-a517d433a52b\") " pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:12.717881 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.717840 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:12.752939 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.752908 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lv6mz"] Jul 22 19:03:12.847983 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.847957 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lv6mz"] Jul 22 19:03:12.850577 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:03:12.850549 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca948a35_f134_4cb8_8906_a517d433a52b.slice/crio-b734036f6fc88703a550ff31fdd85ab43a69fa7e1ca7ec73cf0f325eb86886e8 WatchSource:0}: Error finding container b734036f6fc88703a550ff31fdd85ab43a69fa7e1ca7ec73cf0f325eb86886e8: Status 404 returned error can't find the container with id b734036f6fc88703a550ff31fdd85ab43a69fa7e1ca7ec73cf0f325eb86886e8 Jul 22 19:03:12.961911 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.961830 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kzvht"] Jul 22 19:03:12.966588 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.966566 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:12.973367 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:12.973336 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kzvht"] Jul 22 19:03:13.065436 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:13.065400 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjn66\" (UniqueName: \"kubernetes.io/projected/647acce4-20c3-484e-a712-8a44ac928bb8-kube-api-access-fjn66\") pod \"kuadrant-operator-catalog-kzvht\" (UID: \"647acce4-20c3-484e-a712-8a44ac928bb8\") " pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:13.166672 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:13.166632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjn66\" (UniqueName: \"kubernetes.io/projected/647acce4-20c3-484e-a712-8a44ac928bb8-kube-api-access-fjn66\") pod \"kuadrant-operator-catalog-kzvht\" (UID: \"647acce4-20c3-484e-a712-8a44ac928bb8\") " pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:13.175079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:13.175047 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjn66\" (UniqueName: \"kubernetes.io/projected/647acce4-20c3-484e-a712-8a44ac928bb8-kube-api-access-fjn66\") pod \"kuadrant-operator-catalog-kzvht\" (UID: \"647acce4-20c3-484e-a712-8a44ac928bb8\") " pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:13.258838 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:13.258750 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" event={"ID":"ca948a35-f134-4cb8-8906-a517d433a52b","Type":"ContainerStarted","Data":"b734036f6fc88703a550ff31fdd85ab43a69fa7e1ca7ec73cf0f325eb86886e8"} Jul 22 19:03:13.277077 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:13.277049 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:13.411184 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:13.411159 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kzvht"] Jul 22 19:03:13.434341 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:03:13.434304 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647acce4_20c3_484e_a712_8a44ac928bb8.slice/crio-b90f20d530df0cc79aa67d80312074f836b7e03ddb180390caca6e856fbfabc0 WatchSource:0}: Error finding container b90f20d530df0cc79aa67d80312074f836b7e03ddb180390caca6e856fbfabc0: Status 404 returned error can't find the container with id b90f20d530df0cc79aa67d80312074f836b7e03ddb180390caca6e856fbfabc0 Jul 22 19:03:14.265421 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:14.265381 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" event={"ID":"647acce4-20c3-484e-a712-8a44ac928bb8","Type":"ContainerStarted","Data":"b90f20d530df0cc79aa67d80312074f836b7e03ddb180390caca6e856fbfabc0"} Jul 22 19:03:15.270574 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.270485 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" event={"ID":"647acce4-20c3-484e-a712-8a44ac928bb8","Type":"ContainerStarted","Data":"f223208b12f302d79fc925083c958f655619af0f4f2260845fe02de08dd5ed07"} Jul 22 19:03:15.272008 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.271989 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" event={"ID":"ca948a35-f134-4cb8-8906-a517d433a52b","Type":"ContainerStarted","Data":"97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8"} Jul 22 19:03:15.272077 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.272018 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" podUID="ca948a35-f134-4cb8-8906-a517d433a52b" containerName="registry-server" containerID="cri-o://97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8" gracePeriod=2 Jul 22 19:03:15.291564 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.291524 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" podStartSLOduration=1.7361067650000002 podStartE2EDuration="3.291507558s" totalImagesPullingTime="1.555400793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:15.290235818 +0000 UTC m=+396.016796938" watchObservedRunningTime="2026-07-22 19:03:15.291507558 +0000 UTC m=+396.018068659" Jul 22 19:03:15.309665 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.309620 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" podStartSLOduration=1.172234355 podStartE2EDuration="3.309590867s" totalImagesPullingTime="2.137356512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:15.30758606 +0000 UTC m=+396.034147162" watchObservedRunningTime="2026-07-22 19:03:15.309590867 +0000 UTC m=+396.036151966" Jul 22 19:03:15.511417 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.511389 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:15.589080 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.588988 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqfsb\" (UniqueName: \"kubernetes.io/projected/ca948a35-f134-4cb8-8906-a517d433a52b-kube-api-access-fqfsb\") pod \"ca948a35-f134-4cb8-8906-a517d433a52b\" (UID: \"ca948a35-f134-4cb8-8906-a517d433a52b\") " Jul 22 19:03:15.591141 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.591109 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca948a35-f134-4cb8-8906-a517d433a52b-kube-api-access-fqfsb" (OuterVolumeSpecName: "kube-api-access-fqfsb") pod "ca948a35-f134-4cb8-8906-a517d433a52b" (UID: "ca948a35-f134-4cb8-8906-a517d433a52b"). InnerVolumeSpecName "kube-api-access-fqfsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:03:15.689955 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:15.689920 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqfsb\" (UniqueName: \"kubernetes.io/projected/ca948a35-f134-4cb8-8906-a517d433a52b-kube-api-access-fqfsb\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:03:16.277071 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.276988 2565 generic.go:356] "Generic (PLEG): container finished" podID="ca948a35-f134-4cb8-8906-a517d433a52b" containerID="97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8" exitCode=0 Jul 22 19:03:16.277071 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.277030 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" Jul 22 19:03:16.277504 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.277074 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" event={"ID":"ca948a35-f134-4cb8-8906-a517d433a52b","Type":"ContainerDied","Data":"97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8"} Jul 22 19:03:16.277504 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.277112 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lv6mz" event={"ID":"ca948a35-f134-4cb8-8906-a517d433a52b","Type":"ContainerDied","Data":"b734036f6fc88703a550ff31fdd85ab43a69fa7e1ca7ec73cf0f325eb86886e8"} Jul 22 19:03:16.277504 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.277136 2565 scope.go:117] "RemoveContainer" containerID="97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8" Jul 22 19:03:16.286055 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.286038 2565 scope.go:117] "RemoveContainer" containerID="97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8" Jul 22 19:03:16.286315 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:03:16.286295 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8\": container with ID starting with 97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8 not found: ID does not exist" containerID="97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8" Jul 22 19:03:16.286359 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.286325 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8"} err="failed to get container status \"97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8\": rpc error: code = NotFound desc = could not find container \"97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8\": container with ID starting with 97f3e69d8c40af0d21769a3a70ec9fba8afb71bd7c68c8683c3b2d14888b0ff8 not found: ID does not exist" Jul 22 19:03:16.293521 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.293497 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lv6mz"] Jul 22 19:03:16.297719 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:16.297698 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lv6mz"] Jul 22 19:03:17.812894 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:17.812818 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca948a35-f134-4cb8-8906-a517d433a52b" path="/var/lib/kubelet/pods/ca948a35-f134-4cb8-8906-a517d433a52b/volumes" Jul 22 19:03:23.277649 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:23.277580 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:23.278062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:23.277704 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:23.300327 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:23.300298 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:23.326969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:23.326942 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-kzvht" Jul 22 19:03:44.032637 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.032556 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f"] Jul 22 19:03:44.035073 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.033658 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca948a35-f134-4cb8-8906-a517d433a52b" containerName="registry-server" Jul 22 19:03:44.035073 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.033684 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca948a35-f134-4cb8-8906-a517d433a52b" containerName="registry-server" Jul 22 19:03:44.035073 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.033834 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca948a35-f134-4cb8-8906-a517d433a52b" containerName="registry-server" Jul 22 19:03:44.040653 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.040624 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:44.044036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.044011 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 19:03:44.044352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.044337 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4dk6c\"" Jul 22 19:03:44.044711 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.044691 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f"] Jul 22 19:03:44.115906 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.115869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvmnc\" (UniqueName: \"kubernetes.io/projected/f30038ef-4037-4c39-b698-8e9c05b0cea6-kube-api-access-jvmnc\") pod \"dns-operator-controller-manager-864dd78c4c-bzl2f\" (UID: \"f30038ef-4037-4c39-b698-8e9c05b0cea6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:44.217281 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.217241 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvmnc\" (UniqueName: \"kubernetes.io/projected/f30038ef-4037-4c39-b698-8e9c05b0cea6-kube-api-access-jvmnc\") pod \"dns-operator-controller-manager-864dd78c4c-bzl2f\" (UID: \"f30038ef-4037-4c39-b698-8e9c05b0cea6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:44.233025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.232996 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvmnc\" (UniqueName: \"kubernetes.io/projected/f30038ef-4037-4c39-b698-8e9c05b0cea6-kube-api-access-jvmnc\") pod \"dns-operator-controller-manager-864dd78c4c-bzl2f\" (UID: \"f30038ef-4037-4c39-b698-8e9c05b0cea6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:44.351318 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.351217 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:44.485014 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:44.484980 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f"] Jul 22 19:03:44.487843 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:03:44.487817 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30038ef_4037_4c39_b698_8e9c05b0cea6.slice/crio-f41ca57fab4ff180f7a75f4fa905593ac75788f1db65aefe0d034beaba41d4db WatchSource:0}: Error finding container f41ca57fab4ff180f7a75f4fa905593ac75788f1db65aefe0d034beaba41d4db: Status 404 returned error can't find the container with id f41ca57fab4ff180f7a75f4fa905593ac75788f1db65aefe0d034beaba41d4db Jul 22 19:03:45.393221 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:45.393184 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" event={"ID":"f30038ef-4037-4c39-b698-8e9c05b0cea6","Type":"ContainerStarted","Data":"f41ca57fab4ff180f7a75f4fa905593ac75788f1db65aefe0d034beaba41d4db"} Jul 22 19:03:46.399009 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:46.398974 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" event={"ID":"f30038ef-4037-4c39-b698-8e9c05b0cea6","Type":"ContainerStarted","Data":"08e434e4fa62f00839b17871c4d8867db88f411ba1a23fe8cff6d6592e55e553"} Jul 22 19:03:46.399415 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:46.399094 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:46.417431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:46.417335 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" podStartSLOduration=0.745087518 podStartE2EDuration="2.417319028s" totalImagesPullingTime="1.67223151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:46.416344005 +0000 UTC m=+427.142905108" watchObservedRunningTime="2026-07-22 19:03:46.417319028 +0000 UTC m=+427.143880129" Jul 22 19:03:49.548383 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.548345 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8"] Jul 22 19:03:49.553792 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.553770 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.556421 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.556397 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mwjz4\"" Jul 22 19:03:49.557779 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.557759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.557841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.557811 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdcn6\" (UniqueName: \"kubernetes.io/projected/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-kube-api-access-kdcn6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.567741 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.567711 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8"] Jul 22 19:03:49.658729 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.658678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.658915 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.658773 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdcn6\" (UniqueName: \"kubernetes.io/projected/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-kube-api-access-kdcn6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.659145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.659122 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.675867 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.675842 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdcn6\" (UniqueName: \"kubernetes.io/projected/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-kube-api-access-kdcn6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:49.865829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:49.865726 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:50.025210 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:50.025182 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8"] Jul 22 19:03:50.027278 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:03:50.027253 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76e74a0_f6f6_4e64_9bb0_acd8bb77003f.slice/crio-c56aa724b019d0dc2c8c58b2f98e27c4c298abecf349cb74f260a4665bc23087 WatchSource:0}: Error finding container c56aa724b019d0dc2c8c58b2f98e27c4c298abecf349cb74f260a4665bc23087: Status 404 returned error can't find the container with id c56aa724b019d0dc2c8c58b2f98e27c4c298abecf349cb74f260a4665bc23087 Jul 22 19:03:50.416590 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:50.416557 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" event={"ID":"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f","Type":"ContainerStarted","Data":"c56aa724b019d0dc2c8c58b2f98e27c4c298abecf349cb74f260a4665bc23087"} Jul 22 19:03:54.436687 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:54.436651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" event={"ID":"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f","Type":"ContainerStarted","Data":"63fa0cd86ee3716044cac645d6430a10c2ef9aef62e1767dd07ae78c782f8bdd"} Jul 22 19:03:54.437302 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:54.436737 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:03:54.459001 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:54.458953 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" podStartSLOduration=1.910108674 podStartE2EDuration="5.458934936s" totalImagesPullingTime="3.548826262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:54.456872575 +0000 UTC m=+435.183433678" watchObservedRunningTime="2026-07-22 19:03:54.458934936 +0000 UTC m=+435.185496037" Jul 22 19:03:55.707182 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.707149 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj"] Jul 22 19:03:55.710512 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.710496 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:03:55.713111 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.713092 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-pnmbb\"" Jul 22 19:03:55.730066 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.727160 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj"] Jul 22 19:03:55.809471 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.809425 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7gr5\" (UniqueName: \"kubernetes.io/projected/70685a16-0c64-46ed-8bac-cd573e834559-kube-api-access-h7gr5\") pod \"authorino-operator-75cc8d5cb7-mpjnj\" (UID: \"70685a16-0c64-46ed-8bac-cd573e834559\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:03:55.910860 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.910812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7gr5\" (UniqueName: \"kubernetes.io/projected/70685a16-0c64-46ed-8bac-cd573e834559-kube-api-access-h7gr5\") pod \"authorino-operator-75cc8d5cb7-mpjnj\" (UID: \"70685a16-0c64-46ed-8bac-cd573e834559\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:03:55.926971 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:55.926936 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7gr5\" (UniqueName: \"kubernetes.io/projected/70685a16-0c64-46ed-8bac-cd573e834559-kube-api-access-h7gr5\") pod \"authorino-operator-75cc8d5cb7-mpjnj\" (UID: \"70685a16-0c64-46ed-8bac-cd573e834559\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:03:56.020040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:56.019949 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:03:56.159522 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:56.159499 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj"] Jul 22 19:03:56.161984 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:03:56.161958 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70685a16_0c64_46ed_8bac_cd573e834559.slice/crio-01681c228ff5139071bfa298dbc1a6585dbedd4416e4b5dd9b7b2aa83483ea81 WatchSource:0}: Error finding container 01681c228ff5139071bfa298dbc1a6585dbedd4416e4b5dd9b7b2aa83483ea81: Status 404 returned error can't find the container with id 01681c228ff5139071bfa298dbc1a6585dbedd4416e4b5dd9b7b2aa83483ea81 Jul 22 19:03:56.446272 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:56.446241 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" event={"ID":"70685a16-0c64-46ed-8bac-cd573e834559","Type":"ContainerStarted","Data":"01681c228ff5139071bfa298dbc1a6585dbedd4416e4b5dd9b7b2aa83483ea81"} Jul 22 19:03:57.405429 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:57.405392 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bzl2f" Jul 22 19:03:58.457320 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:58.457286 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" event={"ID":"70685a16-0c64-46ed-8bac-cd573e834559","Type":"ContainerStarted","Data":"3bda4780156a5dd41be2a3640d2dd6580d0daec3617fc44952ef0e541cbd65f7"} Jul 22 19:03:58.457710 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:58.457411 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:03:58.479326 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:03:58.479263 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" podStartSLOduration=1.3984175699999999 podStartE2EDuration="3.479244086s" totalImagesPullingTime="2.080826516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:58.478377137 +0000 UTC m=+439.204938242" watchObservedRunningTime="2026-07-22 19:03:58.479244086 +0000 UTC m=+439.205805188" Jul 22 19:04:05.443455 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:05.443384 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:04:07.320551 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.320514 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8"] Jul 22 19:04:07.320982 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.320724 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" containerName="manager" containerID="cri-o://63fa0cd86ee3716044cac645d6430a10c2ef9aef62e1767dd07ae78c782f8bdd" gracePeriod=2 Jul 22 19:04:07.332832 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.332808 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8"] Jul 22 19:04:07.345963 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:04:07.345901 2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\": pods \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:04:07.348696 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.348673 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b"] Jul 22 19:04:07.349002 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.348986 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" containerName="manager" Jul 22 19:04:07.349072 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.349005 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" containerName="manager" Jul 22 19:04:07.349122 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.349071 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" containerName="manager" Jul 22 19:04:07.351975 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.351953 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.371592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.371565 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b"] Jul 22 19:04:07.386467 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:04:07.386426 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" is forbidden: User \"system:node:ip-10-0-137-176.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-176.ec2.internal' and this object" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:04:07.403394 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.403368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7fg6b\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.403516 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.403417 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbvl2\" (UniqueName: \"kubernetes.io/projected/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-kube-api-access-nbvl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7fg6b\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.498040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.498011 2565 generic.go:356] "Generic (PLEG): container finished" podID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" containerID="63fa0cd86ee3716044cac645d6430a10c2ef9aef62e1767dd07ae78c782f8bdd" exitCode=0 Jul 22 19:04:07.504365 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.504340 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7fg6b\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.504501 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.504388 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvl2\" (UniqueName: \"kubernetes.io/projected/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-kube-api-access-nbvl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7fg6b\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.504788 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.504756 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7fg6b\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.516215 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.516181 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbvl2\" (UniqueName: \"kubernetes.io/projected/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-kube-api-access-nbvl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7fg6b\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.561085 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.561063 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:04:07.564919 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:04:07.564893 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t7qr8\" is forbidden: User \"system:node:ip-10-0-137-176.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-176.ec2.internal' and this object" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:04:07.605391 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.605326 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-extensions-socket-volume\") pod \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " Jul 22 19:04:07.605391 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.605364 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdcn6\" (UniqueName: \"kubernetes.io/projected/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-kube-api-access-kdcn6\") pod \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\" (UID: \"b76e74a0-f6f6-4e64-9bb0-acd8bb77003f\") " Jul 22 19:04:07.605831 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.605806 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" (UID: "b76e74a0-f6f6-4e64-9bb0-acd8bb77003f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:04:07.607365 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.607346 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-kube-api-access-kdcn6" (OuterVolumeSpecName: "kube-api-access-kdcn6") pod "b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" (UID: "b76e74a0-f6f6-4e64-9bb0-acd8bb77003f"). InnerVolumeSpecName "kube-api-access-kdcn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:04:07.700956 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.700918 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:07.705964 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.705948 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-extensions-socket-volume\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:04:07.706017 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.705967 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdcn6\" (UniqueName: \"kubernetes.io/projected/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f-kube-api-access-kdcn6\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:04:07.813462 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.813433 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76e74a0-f6f6-4e64-9bb0-acd8bb77003f" path="/var/lib/kubelet/pods/b76e74a0-f6f6-4e64-9bb0-acd8bb77003f/volumes" Jul 22 19:04:07.837912 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:07.837890 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b"] Jul 22 19:04:07.840156 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:04:07.840137 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1db5d3_6853_4bd1_b56f_7d25552e8ec1.slice/crio-8c3cef14ad88e1a449d27fad28659aff15ce04b37bfae4ed3320fcd85b8c13df WatchSource:0}: Error finding container 8c3cef14ad88e1a449d27fad28659aff15ce04b37bfae4ed3320fcd85b8c13df: Status 404 returned error can't find the container with id 8c3cef14ad88e1a449d27fad28659aff15ce04b37bfae4ed3320fcd85b8c13df Jul 22 19:04:08.503222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:08.503175 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" event={"ID":"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1","Type":"ContainerStarted","Data":"9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f"} Jul 22 19:04:08.503711 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:08.503244 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" event={"ID":"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1","Type":"ContainerStarted","Data":"8c3cef14ad88e1a449d27fad28659aff15ce04b37bfae4ed3320fcd85b8c13df"} Jul 22 19:04:08.503711 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:08.503272 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:08.504386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:08.504370 2565 scope.go:117] "RemoveContainer" containerID="63fa0cd86ee3716044cac645d6430a10c2ef9aef62e1767dd07ae78c782f8bdd" Jul 22 19:04:08.504386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:08.504383 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t7qr8" Jul 22 19:04:08.526959 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:08.526890 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" podStartSLOduration=1.52686924 podStartE2EDuration="1.52686924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:04:08.526513209 +0000 UTC m=+449.253074330" watchObservedRunningTime="2026-07-22 19:04:08.52686924 +0000 UTC m=+449.253430343" Jul 22 19:04:09.463995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:09.463966 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mpjnj" Jul 22 19:04:19.511581 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:19.511551 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:36.178862 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.178829 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b"] Jul 22 19:04:36.179322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.179063 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" podUID="1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" containerName="manager" containerID="cri-o://9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f" gracePeriod=10 Jul 22 19:04:36.423357 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.423334 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:36.541086 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.540998 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbvl2\" (UniqueName: \"kubernetes.io/projected/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-kube-api-access-nbvl2\") pod \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " Jul 22 19:04:36.541234 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.541088 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-extensions-socket-volume\") pod \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\" (UID: \"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1\") " Jul 22 19:04:36.541490 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.541465 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" (UID: "1b1db5d3-6853-4bd1-b56f-7d25552e8ec1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:04:36.543062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.543037 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-kube-api-access-nbvl2" (OuterVolumeSpecName: "kube-api-access-nbvl2") pod "1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" (UID: "1b1db5d3-6853-4bd1-b56f-7d25552e8ec1"). InnerVolumeSpecName "kube-api-access-nbvl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:04:36.625207 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.625174 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" containerID="9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f" exitCode=0 Jul 22 19:04:36.625373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.625217 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" Jul 22 19:04:36.625373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.625243 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" event={"ID":"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1","Type":"ContainerDied","Data":"9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f"} Jul 22 19:04:36.625373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.625284 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b" event={"ID":"1b1db5d3-6853-4bd1-b56f-7d25552e8ec1","Type":"ContainerDied","Data":"8c3cef14ad88e1a449d27fad28659aff15ce04b37bfae4ed3320fcd85b8c13df"} Jul 22 19:04:36.625373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.625304 2565 scope.go:117] "RemoveContainer" containerID="9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f" Jul 22 19:04:36.634479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.634461 2565 scope.go:117] "RemoveContainer" containerID="9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f" Jul 22 19:04:36.634764 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:04:36.634746 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f\": container with ID starting with 9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f not found: ID does not exist" containerID="9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f" Jul 22 19:04:36.634815 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.634773 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f"} err="failed to get container status \"9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f\": rpc error: code = NotFound desc = could not find container \"9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f\": container with ID starting with 9ddcf08aa37466193b39a0e6c29168b8851d5ee9feab12223f73364da9fa9e0f not found: ID does not exist" Jul 22 19:04:36.642163 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.642141 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbvl2\" (UniqueName: \"kubernetes.io/projected/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-kube-api-access-nbvl2\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:04:36.642235 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.642165 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1-extensions-socket-volume\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:04:36.646900 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.646874 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b"] Jul 22 19:04:36.654897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:36.654876 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7fg6b"] Jul 22 19:04:37.813174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:37.813144 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" path="/var/lib/kubelet/pods/1b1db5d3-6853-4bd1-b56f-7d25552e8ec1/volumes" Jul 22 19:04:52.564144 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.564112 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd"] Jul 22 19:04:52.564761 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.564433 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" containerName="manager" Jul 22 19:04:52.564761 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.564445 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" containerName="manager" Jul 22 19:04:52.564761 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.564488 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b1db5d3-6853-4bd1-b56f-7d25552e8ec1" containerName="manager" Jul 22 19:04:52.567725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.567708 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.570512 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.570493 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-m764p\"" Jul 22 19:04:52.584556 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.584532 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd"] Jul 22 19:04:52.679038 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.678992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac6da53a-b447-4006-9def-52442f5e25c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679150 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac6da53a-b447-4006-9def-52442f5e25c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679189 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwjg9\" (UniqueName: \"kubernetes.io/projected/ac6da53a-b447-4006-9def-52442f5e25c9-kube-api-access-hwjg9\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679253 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679567 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac6da53a-b447-4006-9def-52442f5e25c9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.679567 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.679354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.780768 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.780768 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780773 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwjg9\" (UniqueName: \"kubernetes.io/projected/ac6da53a-b447-4006-9def-52442f5e25c9-kube-api-access-hwjg9\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac6da53a-b447-4006-9def-52442f5e25c9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780853 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac6da53a-b447-4006-9def-52442f5e25c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781023 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.780971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.781163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac6da53a-b447-4006-9def-52442f5e25c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.781199 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781538 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.781517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781623 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.781552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781688 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.781624 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.781962 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.781941 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac6da53a-b447-4006-9def-52442f5e25c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.783259 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.783230 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac6da53a-b447-4006-9def-52442f5e25c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.783445 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.783424 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac6da53a-b447-4006-9def-52442f5e25c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.789969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.789943 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac6da53a-b447-4006-9def-52442f5e25c9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.790230 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.790205 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwjg9\" (UniqueName: \"kubernetes.io/projected/ac6da53a-b447-4006-9def-52442f5e25c9-kube-api-access-hwjg9\") pod \"maas-default-gateway-openshift-default-56654dcf7-9cscd\" (UID: \"ac6da53a-b447-4006-9def-52442f5e25c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:52.878059 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:52.877973 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:53.012786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.012715 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd"] Jul 22 19:04:53.016281 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:04:53.016253 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6da53a_b447_4006_9def_52442f5e25c9.slice/crio-ed0656b35f46dd8179a1b9e42fbea6a350890a0d9b3fe9e7d761c9d6cee1b47e WatchSource:0}: Error finding container ed0656b35f46dd8179a1b9e42fbea6a350890a0d9b3fe9e7d761c9d6cee1b47e: Status 404 returned error can't find the container with id ed0656b35f46dd8179a1b9e42fbea6a350890a0d9b3fe9e7d761c9d6cee1b47e Jul 22 19:04:53.018726 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.018688 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:04:53.018800 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.018784 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:04:53.018848 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.018826 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:04:53.696425 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.696392 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" event={"ID":"ac6da53a-b447-4006-9def-52442f5e25c9","Type":"ContainerStarted","Data":"f88cbc8a54529b8397c1edeb656d179e171a33f3c3c2bf8915ded726b3ddf17c"} Jul 22 19:04:53.696816 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.696443 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" event={"ID":"ac6da53a-b447-4006-9def-52442f5e25c9","Type":"ContainerStarted","Data":"ed0656b35f46dd8179a1b9e42fbea6a350890a0d9b3fe9e7d761c9d6cee1b47e"} Jul 22 19:04:53.716935 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.716875 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" podStartSLOduration=1.716853524 podStartE2EDuration="1.716853524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:04:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:04:53.715918646 +0000 UTC m=+494.442479748" watchObservedRunningTime="2026-07-22 19:04:53.716853524 +0000 UTC m=+494.443414627" Jul 22 19:04:53.879041 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:53.879007 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:54.883145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:54.883116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:55.373893 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.373857 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:04:55.376243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.376224 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.378753 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.378722 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7mkgk\"" Jul 22 19:04:55.378753 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.378723 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 19:04:55.386787 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.386757 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:04:55.401441 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.401406 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:04:55.402177 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.402156 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/05187f41-0418-4725-a92c-d30adba3bd03-config-file\") pod \"limitador-limitador-54d6d988d5-xmgc9\" (UID: \"05187f41-0418-4725-a92c-d30adba3bd03\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.402228 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.402198 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfk6q\" (UniqueName: \"kubernetes.io/projected/05187f41-0418-4725-a92c-d30adba3bd03-kube-api-access-cfk6q\") pod \"limitador-limitador-54d6d988d5-xmgc9\" (UID: \"05187f41-0418-4725-a92c-d30adba3bd03\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.502781 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.502745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfk6q\" (UniqueName: \"kubernetes.io/projected/05187f41-0418-4725-a92c-d30adba3bd03-kube-api-access-cfk6q\") pod \"limitador-limitador-54d6d988d5-xmgc9\" (UID: \"05187f41-0418-4725-a92c-d30adba3bd03\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.502951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.502828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/05187f41-0418-4725-a92c-d30adba3bd03-config-file\") pod \"limitador-limitador-54d6d988d5-xmgc9\" (UID: \"05187f41-0418-4725-a92c-d30adba3bd03\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.503477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.503459 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/05187f41-0418-4725-a92c-d30adba3bd03-config-file\") pod \"limitador-limitador-54d6d988d5-xmgc9\" (UID: \"05187f41-0418-4725-a92c-d30adba3bd03\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.511625 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.511574 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfk6q\" (UniqueName: \"kubernetes.io/projected/05187f41-0418-4725-a92c-d30adba3bd03-kube-api-access-cfk6q\") pod \"limitador-limitador-54d6d988d5-xmgc9\" (UID: \"05187f41-0418-4725-a92c-d30adba3bd03\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.688226 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.688196 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:55.706450 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.706424 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:55.707738 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.707715 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:04:55.836428 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:55.836396 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:04:55.839894 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:04:55.839867 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05187f41_0418_4725_a92c_d30adba3bd03.slice/crio-f9aba7e841212667fa0d0f74195db1ff0f4a0b98754ec1a1e0009d9d31430736 WatchSource:0}: Error finding container f9aba7e841212667fa0d0f74195db1ff0f4a0b98754ec1a1e0009d9d31430736: Status 404 returned error can't find the container with id f9aba7e841212667fa0d0f74195db1ff0f4a0b98754ec1a1e0009d9d31430736 Jul 22 19:04:56.714580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:56.714496 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" event={"ID":"05187f41-0418-4725-a92c-d30adba3bd03","Type":"ContainerStarted","Data":"f9aba7e841212667fa0d0f74195db1ff0f4a0b98754ec1a1e0009d9d31430736"} Jul 22 19:04:58.725790 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:58.725757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" event={"ID":"05187f41-0418-4725-a92c-d30adba3bd03","Type":"ContainerStarted","Data":"f47022050ceb15c0f09378ab8b83aeb0bee6d488bafc2a628eaa7032037d49a3"} Jul 22 19:04:58.726198 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:58.725889 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:04:58.744909 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:04:58.744857 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" podStartSLOduration=1.311781956 podStartE2EDuration="3.744839649s" totalImagesPullingTime="2.433057693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:04:58.743046454 +0000 UTC m=+499.469607557" watchObservedRunningTime="2026-07-22 19:04:58.744839649 +0000 UTC m=+499.471400751" Jul 22 19:05:09.731878 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:09.731845 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xmgc9" Jul 22 19:05:24.213010 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.212974 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c6ccd6669-74wqq"] Jul 22 19:05:24.220569 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.220540 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:24.223068 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.223040 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c6ccd6669-74wqq"] Jul 22 19:05:24.223418 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.223400 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5bxpc\"" Jul 22 19:05:24.319838 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.319801 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c6ccd6669-74wqq"] Jul 22 19:05:24.320080 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:05:24.320059 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cn92h], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7c6ccd6669-74wqq" podUID="e707a51d-a01b-4697-8123-c1b66c13d763" Jul 22 19:05:24.342729 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.342688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn92h\" (UniqueName: \"kubernetes.io/projected/e707a51d-a01b-4697-8123-c1b66c13d763-kube-api-access-cn92h\") pod \"authorino-7c6ccd6669-74wqq\" (UID: \"e707a51d-a01b-4697-8123-c1b66c13d763\") " pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:24.347036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.347008 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fb96b6d65-b74zk"] Jul 22 19:05:24.350452 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.350436 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.353517 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.353495 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 19:05:24.361560 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.361537 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fb96b6d65-b74zk"] Jul 22 19:05:24.444062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.444025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn92h\" (UniqueName: \"kubernetes.io/projected/e707a51d-a01b-4697-8123-c1b66c13d763-kube-api-access-cn92h\") pod \"authorino-7c6ccd6669-74wqq\" (UID: \"e707a51d-a01b-4697-8123-c1b66c13d763\") " pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:24.444062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.444077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2gsc\" (UniqueName: \"kubernetes.io/projected/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-kube-api-access-m2gsc\") pod \"authorino-5fb96b6d65-b74zk\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.444292 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.444186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-tls-cert\") pod \"authorino-5fb96b6d65-b74zk\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.452521 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.452497 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn92h\" (UniqueName: \"kubernetes.io/projected/e707a51d-a01b-4697-8123-c1b66c13d763-kube-api-access-cn92h\") pod \"authorino-7c6ccd6669-74wqq\" (UID: \"e707a51d-a01b-4697-8123-c1b66c13d763\") " pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:24.545739 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.545646 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2gsc\" (UniqueName: \"kubernetes.io/projected/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-kube-api-access-m2gsc\") pod \"authorino-5fb96b6d65-b74zk\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.545739 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.545698 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-tls-cert\") pod \"authorino-5fb96b6d65-b74zk\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.548105 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.548081 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-tls-cert\") pod \"authorino-5fb96b6d65-b74zk\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.555185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.555157 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2gsc\" (UniqueName: \"kubernetes.io/projected/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-kube-api-access-m2gsc\") pod \"authorino-5fb96b6d65-b74zk\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.659338 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.659303 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:05:24.788699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.788675 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fb96b6d65-b74zk"] Jul 22 19:05:24.790690 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:05:24.790659 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod302b50fa_8a4b_4d8c_b2ee_21fb54ec526c.slice/crio-5180565a8aa119141de8c162adab1670aa2475f04dbc4bca570cba50582f5131 WatchSource:0}: Error finding container 5180565a8aa119141de8c162adab1670aa2475f04dbc4bca570cba50582f5131: Status 404 returned error can't find the container with id 5180565a8aa119141de8c162adab1670aa2475f04dbc4bca570cba50582f5131 Jul 22 19:05:24.832743 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.832666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:24.832873 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.832670 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" event={"ID":"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c","Type":"ContainerStarted","Data":"5180565a8aa119141de8c162adab1670aa2475f04dbc4bca570cba50582f5131"} Jul 22 19:05:24.841094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.841074 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:24.948576 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.948544 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn92h\" (UniqueName: \"kubernetes.io/projected/e707a51d-a01b-4697-8123-c1b66c13d763-kube-api-access-cn92h\") pod \"e707a51d-a01b-4697-8123-c1b66c13d763\" (UID: \"e707a51d-a01b-4697-8123-c1b66c13d763\") " Jul 22 19:05:24.950580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:24.950554 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e707a51d-a01b-4697-8123-c1b66c13d763-kube-api-access-cn92h" (OuterVolumeSpecName: "kube-api-access-cn92h") pod "e707a51d-a01b-4697-8123-c1b66c13d763" (UID: "e707a51d-a01b-4697-8123-c1b66c13d763"). InnerVolumeSpecName "kube-api-access-cn92h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:05:25.049429 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:25.049382 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn92h\" (UniqueName: \"kubernetes.io/projected/e707a51d-a01b-4697-8123-c1b66c13d763-kube-api-access-cn92h\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:05:25.842562 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:25.841912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c6ccd6669-74wqq" Jul 22 19:05:25.869992 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:25.869964 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c6ccd6669-74wqq"] Jul 22 19:05:25.873527 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:25.873470 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c6ccd6669-74wqq"] Jul 22 19:05:27.813457 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:27.813413 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e707a51d-a01b-4697-8123-c1b66c13d763" path="/var/lib/kubelet/pods/e707a51d-a01b-4697-8123-c1b66c13d763/volumes" Jul 22 19:05:28.857366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:28.857330 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" event={"ID":"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c","Type":"ContainerStarted","Data":"0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0"} Jul 22 19:05:28.872656 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:05:28.872596 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" podStartSLOduration=1.805681745 podStartE2EDuration="4.872582193s" totalImagesPullingTime="3.066900448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:05:28.872482581 +0000 UTC m=+529.599043682" watchObservedRunningTime="2026-07-22 19:05:28.872582193 +0000 UTC m=+529.599143294" Jul 22 19:06:10.890322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:10.890289 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:06:19.071680 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:19.071649 2565 generic.go:356] "Generic (PLEG): container finished" podID="ac6da53a-b447-4006-9def-52442f5e25c9" containerID="f88cbc8a54529b8397c1edeb656d179e171a33f3c3c2bf8915ded726b3ddf17c" exitCode=0 Jul 22 19:06:19.072090 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:19.071698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" event={"ID":"ac6da53a-b447-4006-9def-52442f5e25c9","Type":"ContainerDied","Data":"f88cbc8a54529b8397c1edeb656d179e171a33f3c3c2bf8915ded726b3ddf17c"} Jul 22 19:06:19.072090 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:19.072028 2565 scope.go:117] "RemoveContainer" containerID="f88cbc8a54529b8397c1edeb656d179e171a33f3c3c2bf8915ded726b3ddf17c" Jul 22 19:06:19.073076 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:19.073044 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:06:19.073205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:19.073183 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:06:19.073249 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:19.073232 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:06:20.081416 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:20.081371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" event={"ID":"ac6da53a-b447-4006-9def-52442f5e25c9","Type":"ContainerStarted","Data":"27519de18e75f29a44c509d7a73c7d38697eef914a9c7e50e364ff0b5937a281"} Jul 22 19:06:20.879673 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:20.879638 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:06:20.880712 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:20.880674 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9cscd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 22 19:06:20.880871 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:20.880737 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" podUID="ac6da53a-b447-4006-9def-52442f5e25c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 22 19:06:21.879241 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:21.879193 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9cscd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 22 19:06:21.880069 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:21.880034 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" podUID="ac6da53a-b447-4006-9def-52442f5e25c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 22 19:06:22.879773 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:22.879736 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9cscd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 22 19:06:22.880365 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:22.880333 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" podUID="ac6da53a-b447-4006-9def-52442f5e25c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 22 19:06:23.879072 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:23.879023 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9cscd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 22 19:06:23.879291 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:23.879115 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" podUID="ac6da53a-b447-4006-9def-52442f5e25c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 22 19:06:24.882658 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:24.882628 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:06:24.883045 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:24.882905 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:06:24.883548 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:24.883531 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9cscd" Jul 22 19:06:30.047465 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:30.047429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:06:45.346955 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:45.346916 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:06:50.047019 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:06:50.046986 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:07:01.541443 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.541361 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd"] Jul 22 19:07:01.543860 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.543838 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.546485 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.546462 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 19:07:01.547460 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.547440 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 19:07:01.547709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.547440 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 19:07:01.547709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.547505 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qbfdn\"" Jul 22 19:07:01.553408 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.553379 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.553550 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.553425 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.553550 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.553533 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9571595d-86b0-4375-bd26-13531a186cca-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.553681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.553661 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.553724 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.553703 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.553766 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.553744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t4hx\" (UniqueName: \"kubernetes.io/projected/9571595d-86b0-4375-bd26-13531a186cca-kube-api-access-7t4hx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.557704 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.557679 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd"] Jul 22 19:07:01.654244 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654244 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654248 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654488 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654273 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t4hx\" (UniqueName: \"kubernetes.io/projected/9571595d-86b0-4375-bd26-13531a186cca-kube-api-access-7t4hx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654488 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654304 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654488 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654321 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654488 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654355 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9571595d-86b0-4375-bd26-13531a186cca-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654719 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654656 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654777 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654713 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.654893 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.654821 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.656655 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.656634 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9571595d-86b0-4375-bd26-13531a186cca-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.656835 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.656819 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9571595d-86b0-4375-bd26-13531a186cca-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.662755 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.662729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t4hx\" (UniqueName: \"kubernetes.io/projected/9571595d-86b0-4375-bd26-13531a186cca-kube-api-access-7t4hx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd\" (UID: \"9571595d-86b0-4375-bd26-13531a186cca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.855669 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.855543 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:01.996469 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:01.996443 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd"] Jul 22 19:07:01.998913 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:07:01.998887 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9571595d_86b0_4375_bd26_13531a186cca.slice/crio-064423a76c5ac818c324abbe80c44d3a03e84101c2c3beafd961cb91bff64bdb WatchSource:0}: Error finding container 064423a76c5ac818c324abbe80c44d3a03e84101c2c3beafd961cb91bff64bdb: Status 404 returned error can't find the container with id 064423a76c5ac818c324abbe80c44d3a03e84101c2c3beafd961cb91bff64bdb Jul 22 19:07:02.000250 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:02.000228 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:07:02.262175 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:02.262138 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" event={"ID":"9571595d-86b0-4375-bd26-13531a186cca","Type":"ContainerStarted","Data":"064423a76c5ac818c324abbe80c44d3a03e84101c2c3beafd961cb91bff64bdb"} Jul 22 19:07:02.954963 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:02.954930 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:07:03.268860 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:03.268766 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" event={"ID":"9571595d-86b0-4375-bd26-13531a186cca","Type":"ContainerStarted","Data":"e7b640c338f1f641b664c7755c0af7ba1f3ad83d0a3d252382a6434ee953d192"} Jul 22 19:07:03.288563 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:03.288512 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" podStartSLOduration=1.243832962 podStartE2EDuration="2.288493267s" totalImagesPullingTime="1.044660305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:07:03.286477866 +0000 UTC m=+624.013038968" watchObservedRunningTime="2026-07-22 19:07:03.288493267 +0000 UTC m=+624.015054346" Jul 22 19:07:11.856247 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:11.856214 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:11.856683 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:11.856268 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:11.868991 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:11.868961 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:12.318938 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:12.318905 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd" Jul 22 19:07:16.250121 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:16.250084 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:07:23.954222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:07:23.954179 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:08:15.889468 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:15.889434 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-696b684d48-t22v7"] Jul 22 19:08:15.892687 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:15.892671 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:15.900506 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:15.900478 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-696b684d48-t22v7"] Jul 22 19:08:15.991855 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:15.991817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w67qv\" (UniqueName: \"kubernetes.io/projected/5370e11e-53f9-44ba-8317-b1a39a739901-kube-api-access-w67qv\") pod \"authorino-696b684d48-t22v7\" (UID: \"5370e11e-53f9-44ba-8317-b1a39a739901\") " pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:15.992029 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:15.991887 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5370e11e-53f9-44ba-8317-b1a39a739901-tls-cert\") pod \"authorino-696b684d48-t22v7\" (UID: \"5370e11e-53f9-44ba-8317-b1a39a739901\") " pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:16.092814 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.092770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5370e11e-53f9-44ba-8317-b1a39a739901-tls-cert\") pod \"authorino-696b684d48-t22v7\" (UID: \"5370e11e-53f9-44ba-8317-b1a39a739901\") " pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:16.093021 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.092859 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w67qv\" (UniqueName: \"kubernetes.io/projected/5370e11e-53f9-44ba-8317-b1a39a739901-kube-api-access-w67qv\") pod \"authorino-696b684d48-t22v7\" (UID: \"5370e11e-53f9-44ba-8317-b1a39a739901\") " pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:16.095169 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.095144 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5370e11e-53f9-44ba-8317-b1a39a739901-tls-cert\") pod \"authorino-696b684d48-t22v7\" (UID: \"5370e11e-53f9-44ba-8317-b1a39a739901\") " pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:16.101165 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.101135 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w67qv\" (UniqueName: \"kubernetes.io/projected/5370e11e-53f9-44ba-8317-b1a39a739901-kube-api-access-w67qv\") pod \"authorino-696b684d48-t22v7\" (UID: \"5370e11e-53f9-44ba-8317-b1a39a739901\") " pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:16.201931 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.201893 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696b684d48-t22v7" Jul 22 19:08:16.342034 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.342007 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-696b684d48-t22v7"] Jul 22 19:08:16.347458 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:08:16.347418 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5370e11e_53f9_44ba_8317_b1a39a739901.slice/crio-ed9c4cb4423bafc09e712a0d9bfcbe4fab6c5bf304cd7e61f9c69e0b4f2e1f46 WatchSource:0}: Error finding container ed9c4cb4423bafc09e712a0d9bfcbe4fab6c5bf304cd7e61f9c69e0b4f2e1f46: Status 404 returned error can't find the container with id ed9c4cb4423bafc09e712a0d9bfcbe4fab6c5bf304cd7e61f9c69e0b4f2e1f46 Jul 22 19:08:16.574731 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:16.574641 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-696b684d48-t22v7" event={"ID":"5370e11e-53f9-44ba-8317-b1a39a739901","Type":"ContainerStarted","Data":"ed9c4cb4423bafc09e712a0d9bfcbe4fab6c5bf304cd7e61f9c69e0b4f2e1f46"} Jul 22 19:08:17.584175 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:17.584140 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-696b684d48-t22v7" event={"ID":"5370e11e-53f9-44ba-8317-b1a39a739901","Type":"ContainerStarted","Data":"b9ed0afc32c21455df66289a8f870811b376176f032832a2fd6aca3fb618fc95"} Jul 22 19:08:17.663898 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:17.663828 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-696b684d48-t22v7" podStartSLOduration=2.303305695 podStartE2EDuration="2.66380491s" totalImagesPullingTime="360.499215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:08:17.612701934 +0000 UTC m=+698.339263036" watchObservedRunningTime="2026-07-22 19:08:17.66380491 +0000 UTC m=+698.390366065" Jul 22 19:08:17.664094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:17.664074 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fb96b6d65-b74zk"] Jul 22 19:08:17.664261 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:17.664243 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" podUID="302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" containerName="authorino" containerID="cri-o://0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0" gracePeriod=30 Jul 22 19:08:17.925744 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:17.925713 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:08:18.008728 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.008689 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-tls-cert\") pod \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " Jul 22 19:08:18.008728 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.008731 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2gsc\" (UniqueName: \"kubernetes.io/projected/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-kube-api-access-m2gsc\") pod \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\" (UID: \"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c\") " Jul 22 19:08:18.010837 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.010806 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-kube-api-access-m2gsc" (OuterVolumeSpecName: "kube-api-access-m2gsc") pod "302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" (UID: "302b50fa-8a4b-4d8c-b2ee-21fb54ec526c"). InnerVolumeSpecName "kube-api-access-m2gsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:08:18.018733 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.018703 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" (UID: "302b50fa-8a4b-4d8c-b2ee-21fb54ec526c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:08:18.110086 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.110053 2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-tls-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:08:18.110086 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.110083 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2gsc\" (UniqueName: \"kubernetes.io/projected/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c-kube-api-access-m2gsc\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:08:18.596461 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.591696 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" Jul 22 19:08:18.596461 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.592435 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" event={"ID":"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c","Type":"ContainerDied","Data":"0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0"} Jul 22 19:08:18.596461 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.592494 2565 scope.go:117] "RemoveContainer" containerID="0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0" Jul 22 19:08:18.598210 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.598168 2565 generic.go:356] "Generic (PLEG): container finished" podID="302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" containerID="0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0" exitCode=0 Jul 22 19:08:18.598455 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.598383 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb96b6d65-b74zk" event={"ID":"302b50fa-8a4b-4d8c-b2ee-21fb54ec526c","Type":"ContainerDied","Data":"5180565a8aa119141de8c162adab1670aa2475f04dbc4bca570cba50582f5131"} Jul 22 19:08:18.614593 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.614569 2565 scope.go:117] "RemoveContainer" containerID="0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0" Jul 22 19:08:18.615249 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:08:18.615177 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0\": container with ID starting with 0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0 not found: ID does not exist" containerID="0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0" Jul 22 19:08:18.615249 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.615222 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0"} err="failed to get container status \"0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0\": rpc error: code = NotFound desc = could not find container \"0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0\": container with ID starting with 0497d3b117cb1e54a439ddaf56757f368d1b26a10332e7270aead9f7fcdafeb0 not found: ID does not exist" Jul 22 19:08:18.624709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.624679 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fb96b6d65-b74zk"] Jul 22 19:08:18.629102 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:18.629075 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5fb96b6d65-b74zk"] Jul 22 19:08:19.818110 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:19.818079 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" path="/var/lib/kubelet/pods/302b50fa-8a4b-4d8c-b2ee-21fb54ec526c/volumes" Jul 22 19:08:40.064494 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:40.064406 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:08:48.947148 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:48.947110 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:08:58.045750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:08:58.045713 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:09:07.657457 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:09:07.657413 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:10:47.047665 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:10:47.047628 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:10:57.947361 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:10:57.947325 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:11:35.044842 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:11:35.044754 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:11:44.243240 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:11:44.243206 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:11:54.756549 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:11:54.756518 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:12:03.063681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:12:03.063640 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:12:14.965328 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:12:14.965296 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:12:25.559467 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:12:25.559433 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:12:38.155908 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:12:38.155871 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:12:46.460596 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:12:46.460564 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:13:09.252164 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:13:09.252078 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:13:19.063960 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:13:19.063924 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:13:41.854592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:13:41.854551 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:13:56.567767 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:13:56.567731 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:14:08.568346 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:14:08.568309 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:14:19.052828 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:14:19.052791 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:14:34.957278 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:14:34.957189 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:14:45.076062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:14:45.076027 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:16:34.149858 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:16:34.149827 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:16:46.450888 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:16:46.450855 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:16:55.947563 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:16:55.947525 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:17:07.049288 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:17:07.049253 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:17:19.449141 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:17:19.449099 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:17:29.258659 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:17:29.258622 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:17:37.652016 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:17:37.651922 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:17:47.458260 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:17:47.458219 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:17:59.756008 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:17:59.755972 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:18:09.161632 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:09.161577 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:18:20.247175 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:20.247130 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:18:42.651000 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:42.650962 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:18:52.154877 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:52.154843 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:18:57.065190 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.065156 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj"] Jul 22 19:18:57.065586 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.065458 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" containerName="authorino" Jul 22 19:18:57.065586 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.065468 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" containerName="authorino" Jul 22 19:18:57.065586 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.065528 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="302b50fa-8a4b-4d8c-b2ee-21fb54ec526c" containerName="authorino" Jul 22 19:18:57.068390 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.068375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.071426 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.071392 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e16f7a78-kserve-self-signed-certs\"" Jul 22 19:18:57.103478 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.103452 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj"] Jul 22 19:18:57.156350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.156314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-dshm\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.156350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.156351 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-home\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.156602 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.156376 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tmp-dir\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.156602 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.156400 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tls-certs\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.156602 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.156480 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nh4j\" (UniqueName: \"kubernetes.io/projected/2b02f528-4759-4d0b-84cd-3b61763c0cd5-kube-api-access-7nh4j\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.156602 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.156515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-model-cache\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.257452 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nh4j\" (UniqueName: \"kubernetes.io/projected/2b02f528-4759-4d0b-84cd-3b61763c0cd5-kube-api-access-7nh4j\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.257452 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-model-cache\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.257725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-dshm\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.257725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257525 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-home\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.257725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257544 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tmp-dir\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.257725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tls-certs\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.258001 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.257977 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-model-cache\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.258052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.258028 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-home\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.258109 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.258035 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tmp-dir\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.259883 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.259865 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-dshm\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.260151 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.260134 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tls-certs\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.268517 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.268493 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nh4j\" (UniqueName: \"kubernetes.io/projected/2b02f528-4759-4d0b-84cd-3b61763c0cd5-kube-api-access-7nh4j\") pod \"e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.377526 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.377425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:18:57.510929 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:57.510899 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj"] Jul 22 19:18:57.513332 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:18:57.513306 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b02f528_4759_4d0b_84cd_3b61763c0cd5.slice/crio-d6b92fa9b93b8a0918b54bc410530254bac1db1a7b5460527e246191534144f5 WatchSource:0}: Error finding container d6b92fa9b93b8a0918b54bc410530254bac1db1a7b5460527e246191534144f5: Status 404 returned error can't find the container with id d6b92fa9b93b8a0918b54bc410530254bac1db1a7b5460527e246191534144f5 Jul 22 19:18:58.367833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:58.367800 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" event={"ID":"2b02f528-4759-4d0b-84cd-3b61763c0cd5","Type":"ContainerStarted","Data":"aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f"} Jul 22 19:18:58.367833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:58.367839 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" event={"ID":"2b02f528-4759-4d0b-84cd-3b61763c0cd5","Type":"ContainerStarted","Data":"d6b92fa9b93b8a0918b54bc410530254bac1db1a7b5460527e246191534144f5"} Jul 22 19:18:58.391460 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:18:58.391406 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" podStartSLOduration=1.391386968 podStartE2EDuration="1.391386968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:18:58.388845404 +0000 UTC m=+1339.115406507" watchObservedRunningTime="2026-07-22 19:18:58.391386968 +0000 UTC m=+1339.117948069" Jul 22 19:19:01.456992 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:01.456909 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:19:07.378162 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:07.378107 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:19:07.378642 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:07.378192 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:19:07.390727 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:07.390704 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:19:07.421436 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:07.421411 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:19:26.353267 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.353235 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br"] Jul 22 19:19:26.356801 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.356780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.359440 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.359422 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e16f7a78-kserve-self-signed-certs\"" Jul 22 19:19:26.366975 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.366954 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br"] Jul 22 19:19:26.403112 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.403075 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-dshm\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.403112 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.403110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tls-certs\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.403379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.403175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-model-cache\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.403379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.403230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9zs\" (UniqueName: \"kubernetes.io/projected/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-kube-api-access-4l9zs\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.403379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.403297 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tmp-dir\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.403379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.403333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-home\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.504700 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.504671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-dshm\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.504700 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.504705 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tls-certs\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.504955 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.504746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-model-cache\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.504955 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.504779 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9zs\" (UniqueName: \"kubernetes.io/projected/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-kube-api-access-4l9zs\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.505064 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.504948 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tmp-dir\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.505064 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.505000 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-home\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.505213 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.505193 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-model-cache\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.505282 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.505262 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tmp-dir\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.505356 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.505336 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-home\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.506976 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.506960 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-dshm\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.507371 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.507350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tls-certs\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.513895 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.513870 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9zs\" (UniqueName: \"kubernetes.io/projected/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-kube-api-access-4l9zs\") pod \"e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.666288 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.666250 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:26.804362 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:26.804313 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br"] Jul 22 19:19:26.806772 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:19:26.806729 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode870ac08_d44a_4c1f_8b57_3dde6c4d5fac.slice/crio-7f1d7bc5491421f35bfb10714db38f8659eaa10147eb28748d68d287d1bf8af4 WatchSource:0}: Error finding container 7f1d7bc5491421f35bfb10714db38f8659eaa10147eb28748d68d287d1bf8af4: Status 404 returned error can't find the container with id 7f1d7bc5491421f35bfb10714db38f8659eaa10147eb28748d68d287d1bf8af4 Jul 22 19:19:27.503424 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:27.503391 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" event={"ID":"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac","Type":"ContainerStarted","Data":"d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781"} Jul 22 19:19:27.503424 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:27.503431 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" event={"ID":"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac","Type":"ContainerStarted","Data":"7f1d7bc5491421f35bfb10714db38f8659eaa10147eb28748d68d287d1bf8af4"} Jul 22 19:19:27.519990 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:27.519939 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" podStartSLOduration=1.5199231850000001 podStartE2EDuration="1.519923185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:19:27.518772926 +0000 UTC m=+1368.245334029" watchObservedRunningTime="2026-07-22 19:19:27.519923185 +0000 UTC m=+1368.246484287" Jul 22 19:19:29.838977 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:29.838942 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:19:36.667150 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:36.667109 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:36.667786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:36.667166 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:36.679990 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:36.679965 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:37.558438 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:37.558412 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:19:48.199354 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:48.199320 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj"] Jul 22 19:19:49.830138 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:49.830102 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br"] Jul 22 19:19:52.645145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:19:52.645107 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:20:03.221053 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.220978 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" podUID="2b02f528-4759-4d0b-84cd-3b61763c0cd5" containerName="main" containerID="cri-o://aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f" gracePeriod=45 Jul 22 19:20:03.473477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.473414 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:20:03.638696 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638641 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tmp-dir\") pod \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " Jul 22 19:20:03.638696 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638702 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-model-cache\") pod \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " Jul 22 19:20:03.638945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638747 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-dshm\") pod \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " Jul 22 19:20:03.638945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638775 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nh4j\" (UniqueName: \"kubernetes.io/projected/2b02f528-4759-4d0b-84cd-3b61763c0cd5-kube-api-access-7nh4j\") pod \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " Jul 22 19:20:03.638945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638805 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-home\") pod \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " Jul 22 19:20:03.638945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638871 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tls-certs\") pod \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\" (UID: \"2b02f528-4759-4d0b-84cd-3b61763c0cd5\") " Jul 22 19:20:03.639152 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.638921 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b02f528-4759-4d0b-84cd-3b61763c0cd5" (UID: "2b02f528-4759-4d0b-84cd-3b61763c0cd5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:03.639152 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.639011 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b02f528-4759-4d0b-84cd-3b61763c0cd5" (UID: "2b02f528-4759-4d0b-84cd-3b61763c0cd5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:03.639152 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.639117 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-home" (OuterVolumeSpecName: "home") pod "2b02f528-4759-4d0b-84cd-3b61763c0cd5" (UID: "2b02f528-4759-4d0b-84cd-3b61763c0cd5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:03.639308 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.639234 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-home\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:03.639308 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.639255 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tmp-dir\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:03.639308 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.639270 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-model-cache\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:03.641002 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.640970 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-dshm" (OuterVolumeSpecName: "dshm") pod "2b02f528-4759-4d0b-84cd-3b61763c0cd5" (UID: "2b02f528-4759-4d0b-84cd-3b61763c0cd5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:03.641115 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.640996 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b02f528-4759-4d0b-84cd-3b61763c0cd5" (UID: "2b02f528-4759-4d0b-84cd-3b61763c0cd5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:20:03.641115 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.641004 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b02f528-4759-4d0b-84cd-3b61763c0cd5-kube-api-access-7nh4j" (OuterVolumeSpecName: "kube-api-access-7nh4j") pod "2b02f528-4759-4d0b-84cd-3b61763c0cd5" (UID: "2b02f528-4759-4d0b-84cd-3b61763c0cd5"). InnerVolumeSpecName "kube-api-access-7nh4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:20:03.658366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.658332 2565 generic.go:356] "Generic (PLEG): container finished" podID="2b02f528-4759-4d0b-84cd-3b61763c0cd5" containerID="aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f" exitCode=0 Jul 22 19:20:03.658529 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.658379 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" Jul 22 19:20:03.658529 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.658405 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" event={"ID":"2b02f528-4759-4d0b-84cd-3b61763c0cd5","Type":"ContainerDied","Data":"aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f"} Jul 22 19:20:03.658529 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.658440 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj" event={"ID":"2b02f528-4759-4d0b-84cd-3b61763c0cd5","Type":"ContainerDied","Data":"d6b92fa9b93b8a0918b54bc410530254bac1db1a7b5460527e246191534144f5"} Jul 22 19:20:03.658529 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.658456 2565 scope.go:117] "RemoveContainer" containerID="aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f" Jul 22 19:20:03.667420 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.667401 2565 scope.go:117] "RemoveContainer" containerID="aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f" Jul 22 19:20:03.667764 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:20:03.667739 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f\": container with ID starting with aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f not found: ID does not exist" containerID="aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f" Jul 22 19:20:03.667868 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.667774 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f"} err="failed to get container status \"aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f\": rpc error: code = NotFound desc = could not find container \"aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f\": container with ID starting with aa986dcc7f0a2df7bb934c01b25f7079ec672cdda44d0a3c36ffab4d8bb7462f not found: ID does not exist" Jul 22 19:20:03.678492 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.678463 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj"] Jul 22 19:20:03.681372 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.681349 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e16f7a78-kserve-787998bf6c-lmkgj"] Jul 22 19:20:03.740725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.740637 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b02f528-4759-4d0b-84cd-3b61763c0cd5-dshm\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:03.740725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.740665 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nh4j\" (UniqueName: \"kubernetes.io/projected/2b02f528-4759-4d0b-84cd-3b61763c0cd5-kube-api-access-7nh4j\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:03.740725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.740676 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b02f528-4759-4d0b-84cd-3b61763c0cd5-tls-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:03.813775 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:03.813746 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b02f528-4759-4d0b-84cd-3b61763c0cd5" path="/var/lib/kubelet/pods/2b02f528-4759-4d0b-84cd-3b61763c0cd5/volumes" Jul 22 19:20:04.850165 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:04.850085 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" podUID="e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" containerName="main" containerID="cri-o://d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781" gracePeriod=45 Jul 22 19:20:05.105075 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.105013 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:20:05.149553 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149525 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tmp-dir\") pod \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " Jul 22 19:20:05.149553 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149555 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-home\") pod \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " Jul 22 19:20:05.149817 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149575 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-dshm\") pod \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " Jul 22 19:20:05.149817 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149591 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-model-cache\") pod \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " Jul 22 19:20:05.149817 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149653 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l9zs\" (UniqueName: \"kubernetes.io/projected/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-kube-api-access-4l9zs\") pod \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " Jul 22 19:20:05.149817 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149726 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tls-certs\") pod \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\" (UID: \"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac\") " Jul 22 19:20:05.150019 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149868 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" (UID: "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:05.150019 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149878 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-home" (OuterVolumeSpecName: "home") pod "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" (UID: "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:05.150019 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149980 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tmp-dir\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:05.150019 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.149998 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-home\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:05.150207 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.150063 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-model-cache" (OuterVolumeSpecName: "model-cache") pod "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" (UID: "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:05.151704 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.151680 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-dshm" (OuterVolumeSpecName: "dshm") pod "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" (UID: "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:20:05.151943 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.151915 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" (UID: "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:20:05.151943 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.151925 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-kube-api-access-4l9zs" (OuterVolumeSpecName: "kube-api-access-4l9zs") pod "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" (UID: "e870ac08-d44a-4c1f-8b57-3dde6c4d5fac"). InnerVolumeSpecName "kube-api-access-4l9zs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:20:05.250721 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.250685 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-dshm\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:05.250721 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.250716 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-model-cache\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:05.250721 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.250727 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l9zs\" (UniqueName: \"kubernetes.io/projected/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-kube-api-access-4l9zs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:05.251005 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.250736 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac-tls-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:20:05.674805 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.674772 2565 generic.go:356] "Generic (PLEG): container finished" podID="e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" containerID="d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781" exitCode=0 Jul 22 19:20:05.675012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.674813 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" event={"ID":"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac","Type":"ContainerDied","Data":"d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781"} Jul 22 19:20:05.675012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.674820 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" Jul 22 19:20:05.675012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.674840 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br" event={"ID":"e870ac08-d44a-4c1f-8b57-3dde6c4d5fac","Type":"ContainerDied","Data":"7f1d7bc5491421f35bfb10714db38f8659eaa10147eb28748d68d287d1bf8af4"} Jul 22 19:20:05.675012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.674857 2565 scope.go:117] "RemoveContainer" containerID="d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781" Jul 22 19:20:05.683530 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.683513 2565 scope.go:117] "RemoveContainer" containerID="d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781" Jul 22 19:20:05.683820 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:20:05.683803 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781\": container with ID starting with d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781 not found: ID does not exist" containerID="d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781" Jul 22 19:20:05.683870 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.683829 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781"} err="failed to get container status \"d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781\": rpc error: code = NotFound desc = could not find container \"d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781\": container with ID starting with d4dfe4b7a8285f8887eb8df921ce1b3441e4d1c184b9c9cc894a70a2ffb15781 not found: ID does not exist" Jul 22 19:20:05.694402 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.694377 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br"] Jul 22 19:20:05.702187 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.702168 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e16f7a78-kserve-557c7c9999-fs4br"] Jul 22 19:20:05.813255 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:05.813214 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" path="/var/lib/kubelet/pods/e870ac08-d44a-4c1f-8b57-3dde6c4d5fac/volumes" Jul 22 19:20:06.643809 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:06.643775 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:20:13.339274 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:13.339243 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:20:30.350821 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:30.350769 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:20:39.843789 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:39.843759 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:20:49.742311 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:20:49.742278 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:21:00.548060 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:21:00.548022 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:21:10.346651 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:21:10.346601 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:21:19.943246 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:21:19.943200 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:21:33.743867 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:21:33.743821 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:21:48.642913 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:21:48.642873 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:21:58.141601 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:21:58.141561 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:22:05.242923 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:22:05.242839 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:22:15.044371 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:22:15.044337 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:22:22.339177 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:22:22.339144 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:22:39.403577 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:22:39.403534 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:22:51.652082 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:22:51.652048 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:23:03.241556 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:03.241517 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:23:17.243442 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:17.243407 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:23:31.356452 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:31.356368 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:23:36.931460 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931430 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p"] Jul 22 19:23:36.931957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931791 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" containerName="main" Jul 22 19:23:36.931957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931803 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" containerName="main" Jul 22 19:23:36.931957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931823 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b02f528-4759-4d0b-84cd-3b61763c0cd5" containerName="main" Jul 22 19:23:36.931957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931828 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b02f528-4759-4d0b-84cd-3b61763c0cd5" containerName="main" Jul 22 19:23:36.931957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931880 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b02f528-4759-4d0b-84cd-3b61763c0cd5" containerName="main" Jul 22 19:23:36.931957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.931888 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e870ac08-d44a-4c1f-8b57-3dde6c4d5fac" containerName="main" Jul 22 19:23:36.934793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.934776 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:36.937220 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.937192 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cd4f9eba-openshift-default-dockercfg-l7cnn\"" Jul 22 19:23:36.955484 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:36.955445 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p"] Jul 22 19:23:37.019822 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.019787 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-podinfo\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.019822 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.019836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-certs\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.019918 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6srs\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-kube-api-access-q6srs\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.019951 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-credential-socket\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.019992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-token\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.020023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-socket\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.020055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-data\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020279 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.020081 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-envoy\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.020279 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.020106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24e5efec-4229-494e-be61-1c2b7ffc05ac-istiod-ca-cert\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121234 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-podinfo\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121237 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-certs\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121278 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6srs\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-kube-api-access-q6srs\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121305 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-credential-socket\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-token\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-socket\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-data\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121764 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-envoy\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121764 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121476 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24e5efec-4229-494e-be61-1c2b7ffc05ac-istiod-ca-cert\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.121865 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.121844 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-credential-socket\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.122039 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.122013 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-socket\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.122169 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.122091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-certs\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.122169 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.122114 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-data\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.122344 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.122319 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24e5efec-4229-494e-be61-1c2b7ffc05ac-istiod-ca-cert\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.123872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.123850 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-envoy\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.123964 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.123908 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-podinfo\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.129816 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.129788 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-token\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.129930 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.129881 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6srs\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-kube-api-access-q6srs\") pod \"e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.248131 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.248011 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:37.386683 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.386650 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p"] Jul 22 19:23:37.389995 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:23:37.389967 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e5efec_4229_494e_be61_1c2b7ffc05ac.slice/crio-e6d6cf0551c95bd6303f2a03b75eacb98ad6205086fc5ab0b98c52e450b4a68d WatchSource:0}: Error finding container e6d6cf0551c95bd6303f2a03b75eacb98ad6205086fc5ab0b98c52e450b4a68d: Status 404 returned error can't find the container with id e6d6cf0551c95bd6303f2a03b75eacb98ad6205086fc5ab0b98c52e450b4a68d Jul 22 19:23:37.392592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.392559 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:37.392698 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.392646 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:37.392698 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.392679 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:37.596829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.596703 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" event={"ID":"24e5efec-4229-494e-be61-1c2b7ffc05ac","Type":"ContainerStarted","Data":"0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103"} Jul 22 19:23:37.596829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.596779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" event={"ID":"24e5efec-4229-494e-be61-1c2b7ffc05ac","Type":"ContainerStarted","Data":"e6d6cf0551c95bd6303f2a03b75eacb98ad6205086fc5ab0b98c52e450b4a68d"} Jul 22 19:23:37.617424 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:37.617363 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" podStartSLOduration=1.617345494 podStartE2EDuration="1.617345494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:23:37.616105076 +0000 UTC m=+1618.342666178" watchObservedRunningTime="2026-07-22 19:23:37.617345494 +0000 UTC m=+1618.343906596" Jul 22 19:23:38.248475 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:38.248434 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:38.253823 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:38.253797 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:38.601710 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:38.601628 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:38.602803 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:38.602783 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:41.939979 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:41.939946 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:23:50.654716 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:50.654684 2565 generic.go:356] "Generic (PLEG): container finished" podID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerID="0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103" exitCode=0 Jul 22 19:23:50.655268 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:50.654770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" event={"ID":"24e5efec-4229-494e-be61-1c2b7ffc05ac","Type":"ContainerDied","Data":"0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103"} Jul 22 19:23:50.655268 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:50.655141 2565 scope.go:117] "RemoveContainer" containerID="0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103" Jul 22 19:23:50.656176 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:50.656143 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:50.656252 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:50.656233 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:50.656302 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:50.656286 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:51.660587 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:51.660554 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" event={"ID":"24e5efec-4229-494e-be61-1c2b7ffc05ac","Type":"ContainerStarted","Data":"ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373"} Jul 22 19:23:52.248625 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:52.248571 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:52.249572 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:52.249546 2565 patch_prober.go:28] interesting pod/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 22 19:23:52.249721 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:52.249601 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 22 19:23:53.249053 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:53.249010 2565 patch_prober.go:28] interesting pod/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 22 19:23:53.249459 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:53.249072 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 22 19:23:53.602286 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:53.602192 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:54.248781 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:54.248736 2565 patch_prober.go:28] interesting pod/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 22 19:23:54.249001 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:54.248807 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 22 19:23:55.248386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:55.248352 2565 patch_prober.go:28] interesting pod/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 22 19:23:55.248829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:55.248419 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 22 19:23:56.252970 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:56.252936 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:23:56.253983 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:23:56.253964 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:24:18.146452 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:18.146414 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:24:20.157013 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:20.156972 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p"] Jul 22 19:24:20.157499 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:20.157241 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" containerID="cri-o://ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373" gracePeriod=30 Jul 22 19:24:21.284123 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.284094 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf"] Jul 22 19:24:21.287268 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.287251 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.289721 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.289696 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6058163f-gw-openshift-default-dockercfg-29f7k\"" Jul 22 19:24:21.294919 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.294894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-data\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295042 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.294926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe79c4ee-010a-45f9-9aac-6e56410702d7-istiod-ca-cert\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295042 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.294981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-credential-socket\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295042 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.295003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-socket\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295170 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.295058 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-token\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295170 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.295099 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-certs\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295170 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.295164 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-podinfo\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295300 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.295201 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs8n\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-kube-api-access-tcs8n\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.295300 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.295253 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-envoy\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.300536 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.300511 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf"] Jul 22 19:24:21.396313 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396270 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-data\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396313 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396313 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe79c4ee-010a-45f9-9aac-6e56410702d7-istiod-ca-cert\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-credential-socket\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-socket\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-token\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-certs\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-podinfo\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs8n\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-kube-api-access-tcs8n\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396571 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-envoy\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396970 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-data\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.396970 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.396938 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-socket\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.397083 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.397001 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-certs\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.397199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.397179 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-credential-socket\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.397237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.397213 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe79c4ee-010a-45f9-9aac-6e56410702d7-istiod-ca-cert\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.399390 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.399360 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-envoy\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.399591 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.399573 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-podinfo\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.404326 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.404298 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-token\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.404436 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.404419 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs8n\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-kube-api-access-tcs8n\") pod \"e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.598401 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.598294 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:21.739333 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.739302 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf"] Jul 22 19:24:21.742127 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:24:21.742097 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe79c4ee_010a_45f9_9aac_6e56410702d7.slice/crio-168c91ed3ac949b54779ce5ab751a990d94e860a5535c43ab982e4894eeac017 WatchSource:0}: Error finding container 168c91ed3ac949b54779ce5ab751a990d94e860a5535c43ab982e4894eeac017: Status 404 returned error can't find the container with id 168c91ed3ac949b54779ce5ab751a990d94e860a5535c43ab982e4894eeac017 Jul 22 19:24:21.744351 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.744321 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:21.744416 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.744385 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:21.744455 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.744418 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:21.812920 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.812887 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" event={"ID":"fe79c4ee-010a-45f9-9aac-6e56410702d7","Type":"ContainerStarted","Data":"3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66"} Jul 22 19:24:21.813072 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.812941 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" event={"ID":"fe79c4ee-010a-45f9-9aac-6e56410702d7","Type":"ContainerStarted","Data":"168c91ed3ac949b54779ce5ab751a990d94e860a5535c43ab982e4894eeac017"} Jul 22 19:24:21.828360 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:21.828292 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podStartSLOduration=0.828271844 podStartE2EDuration="828.271844ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:21.82525105 +0000 UTC m=+1662.551812152" watchObservedRunningTime="2026-07-22 19:24:21.828271844 +0000 UTC m=+1662.554832946" Jul 22 19:24:22.599360 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:22.599310 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:22.604195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:22.604167 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:22.813136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:22.813103 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:22.814095 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:22.814075 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:25.418829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.418768 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:24:25.430800 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.430769 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24e5efec-4229-494e-be61-1c2b7ffc05ac-istiod-ca-cert\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.430979 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.430818 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-credential-socket\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.430979 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.430856 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-data\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431092 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431037 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-certs\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431142 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431097 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-envoy\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431142 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431121 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:25.431237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431176 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-token\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431209 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6srs\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-kube-api-access-q6srs\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431324 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431247 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-socket\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431324 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431278 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:25.431324 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431296 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-podinfo\") pod \"24e5efec-4229-494e-be61-1c2b7ffc05ac\" (UID: \"24e5efec-4229-494e-be61-1c2b7ffc05ac\") " Jul 22 19:24:25.431456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431311 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24e5efec-4229-494e-be61-1c2b7ffc05ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:24:25.431570 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431545 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:25.431724 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431586 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.431724 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431618 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.431724 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431632 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24e5efec-4229-494e-be61-1c2b7ffc05ac-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.431724 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.431716 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:25.433628 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.433591 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:24:25.433869 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.433847 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:25.433934 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.433871 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-kube-api-access-q6srs" (OuterVolumeSpecName: "kube-api-access-q6srs") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "kube-api-access-q6srs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:25.433934 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.433896 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "24e5efec-4229-494e-be61-1c2b7ffc05ac" (UID: "24e5efec-4229-494e-be61-1c2b7ffc05ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:25.532199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.532164 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.532199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.532194 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.532199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.532203 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.532450 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.532212 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6srs\" (UniqueName: \"kubernetes.io/projected/24e5efec-4229-494e-be61-1c2b7ffc05ac-kube-api-access-q6srs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.532450 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.532232 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24e5efec-4229-494e-be61-1c2b7ffc05ac-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.532450 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.532240 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24e5efec-4229-494e-be61-1c2b7ffc05ac-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:24:25.827934 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.827905 2565 generic.go:356] "Generic (PLEG): container finished" podID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerID="ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373" exitCode=0 Jul 22 19:24:25.828062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.827960 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" Jul 22 19:24:25.828062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.827989 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" event={"ID":"24e5efec-4229-494e-be61-1c2b7ffc05ac","Type":"ContainerDied","Data":"ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373"} Jul 22 19:24:25.828062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.828042 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p" event={"ID":"24e5efec-4229-494e-be61-1c2b7ffc05ac","Type":"ContainerDied","Data":"e6d6cf0551c95bd6303f2a03b75eacb98ad6205086fc5ab0b98c52e450b4a68d"} Jul 22 19:24:25.828162 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.828063 2565 scope.go:117] "RemoveContainer" containerID="ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373" Jul 22 19:24:25.836312 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.836298 2565 scope.go:117] "RemoveContainer" containerID="0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103" Jul 22 19:24:25.845107 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.845074 2565 scope.go:117] "RemoveContainer" containerID="ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373" Jul 22 19:24:25.848998 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.848969 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p"] Jul 22 19:24:25.849843 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:24:25.849818 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373\": container with ID starting with ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373 not found: ID does not exist" containerID="ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373" Jul 22 19:24:25.849961 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.849850 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373"} err="failed to get container status \"ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373\": rpc error: code = NotFound desc = could not find container \"ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373\": container with ID starting with ae2699e5519d36e187c3b3c07812d6dc0fc4b5cfd88d8398afc90e09aaa8d373 not found: ID does not exist" Jul 22 19:24:25.849961 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.849874 2565 scope.go:117] "RemoveContainer" containerID="0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103" Jul 22 19:24:25.850363 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:24:25.850342 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103\": container with ID starting with 0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103 not found: ID does not exist" containerID="0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103" Jul 22 19:24:25.850445 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.850366 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103"} err="failed to get container status \"0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103\": rpc error: code = NotFound desc = could not find container \"0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103\": container with ID starting with 0fe0fcb510f1cc749eeab827ae615f49c0044270175541ffa0de1937914a1103 not found: ID does not exist" Jul 22 19:24:25.850445 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:25.850414 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cd4f9eba-openshift-default-764868b755-zxd9p"] Jul 22 19:24:27.813380 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:27.813342 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" path="/var/lib/kubelet/pods/24e5efec-4229-494e-be61-1c2b7ffc05ac/volumes" Jul 22 19:24:31.349064 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:31.349030 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:24:38.854836 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:38.854787 2565 patch_prober.go:28] interesting pod/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:24:38.855329 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:38.854853 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:24:39.900840 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:39.900808 2565 generic.go:356] "Generic (PLEG): container finished" podID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerID="3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66" exitCode=0 Jul 22 19:24:39.900840 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:39.900846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" event={"ID":"fe79c4ee-010a-45f9-9aac-6e56410702d7","Type":"ContainerDied","Data":"3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66"} Jul 22 19:24:39.901288 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:39.901226 2565 scope.go:117] "RemoveContainer" containerID="3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66" Jul 22 19:24:39.902199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:39.902166 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:39.902352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:39.902249 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:39.902352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:39.902298 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:40.906925 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:40.906889 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" event={"ID":"fe79c4ee-010a-45f9-9aac-6e56410702d7","Type":"ContainerStarted","Data":"b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c"} Jul 22 19:24:41.599082 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:41.599038 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:41.599281 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:41.599262 2565 patch_prober.go:28] interesting pod/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 22 19:24:41.599349 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:41.599308 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 22 19:24:42.598926 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:42.598887 2565 patch_prober.go:28] interesting pod/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 22 19:24:42.599342 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:42.598957 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 22 19:24:43.599557 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:43.599517 2565 patch_prober.go:28] interesting pod/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 22 19:24:43.599980 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:43.599579 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 22 19:24:44.599624 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:44.599583 2565 patch_prober.go:28] interesting pod/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 22 19:24:44.600020 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:44.599665 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 22 19:24:45.602785 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:45.602753 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:45.603275 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:45.603131 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:24:45.603779 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:24:45.603760 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:25:01.942430 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:01.942345 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:25:03.712236 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:03.712202 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf"] Jul 22 19:25:03.712704 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:03.712462 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" containerID="cri-o://b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c" gracePeriod=30 Jul 22 19:25:04.677378 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677344 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v"] Jul 22 19:25:04.677693 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677679 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" Jul 22 19:25:04.677693 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677693 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" Jul 22 19:25:04.677808 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677705 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" Jul 22 19:25:04.677808 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677711 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" Jul 22 19:25:04.677808 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677780 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" Jul 22 19:25:04.677808 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.677787 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="24e5efec-4229-494e-be61-1c2b7ffc05ac" containerName="istio-proxy" Jul 22 19:25:04.681048 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.681027 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.683667 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.683644 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9c70fc9e-openshift-default-dockercfg-7lmjq\"" Jul 22 19:25:04.691016 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.690982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-envoy\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691138 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22h6p\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-kube-api-access-22h6p\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691149 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-token\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691209 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-credential-socket\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691391 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-podinfo\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691391 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-socket\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691521 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691398 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istiod-ca-cert\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691521 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-data\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.691521 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.691495 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-certs\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.693239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.693216 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v"] Jul 22 19:25:04.792338 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792290 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-credential-socket\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-podinfo\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792386 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-socket\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792417 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istiod-ca-cert\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-data\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792493 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-certs\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-envoy\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792578 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22h6p\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-kube-api-access-22h6p\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-token\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.792833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.792759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-credential-socket\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.793306 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.793036 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-data\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.793523 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.793501 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-certs\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.793767 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.793719 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-socket\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.794240 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.794217 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istiod-ca-cert\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.795993 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.795974 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-envoy\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.796276 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.796257 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-podinfo\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.801412 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.801392 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-token\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.801550 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.801531 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22h6p\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-kube-api-access-22h6p\") pod \"e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:04.993639 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:04.993520 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:05.132796 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:05.132772 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v"] Jul 22 19:25:05.134820 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:25:05.134789 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d85184_cf32_400a_9bf8_8d97fb4dd27f.slice/crio-baeddeb6d352dda1e4e4b7971c6b5d9fae72892d324ea9715c78cdeaa0f23a6e WatchSource:0}: Error finding container baeddeb6d352dda1e4e4b7971c6b5d9fae72892d324ea9715c78cdeaa0f23a6e: Status 404 returned error can't find the container with id baeddeb6d352dda1e4e4b7971c6b5d9fae72892d324ea9715c78cdeaa0f23a6e Jul 22 19:25:05.136878 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:05.136843 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:05.136989 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:05.136924 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:05.136989 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:05.136968 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:06.016010 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:06.015976 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" event={"ID":"e8d85184-cf32-400a-9bf8-8d97fb4dd27f","Type":"ContainerStarted","Data":"5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f"} Jul 22 19:25:06.016409 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:06.016028 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" event={"ID":"e8d85184-cf32-400a-9bf8-8d97fb4dd27f","Type":"ContainerStarted","Data":"baeddeb6d352dda1e4e4b7971c6b5d9fae72892d324ea9715c78cdeaa0f23a6e"} Jul 22 19:25:06.035899 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:06.035671 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" podStartSLOduration=2.035653925 podStartE2EDuration="2.035653925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:25:06.03518314 +0000 UTC m=+1706.761744242" watchObservedRunningTime="2026-07-22 19:25:06.035653925 +0000 UTC m=+1706.762215028" Jul 22 19:25:06.993736 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:06.993696 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:06.998884 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:06.998852 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:07.021121 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:07.021084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:07.022016 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:07.021993 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:08.972468 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:08.972409 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:25:09.030663 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030633 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-token\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.030864 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030696 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcs8n\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-kube-api-access-tcs8n\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.030864 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030754 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-credential-socket\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.030864 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030794 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe79c4ee-010a-45f9-9aac-6e56410702d7-istiod-ca-cert\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.030864 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030861 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-envoy\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.031094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030890 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-data\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.031094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030916 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-podinfo\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.031094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030948 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-socket\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.031094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.030969 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-certs\") pod \"fe79c4ee-010a-45f9-9aac-6e56410702d7\" (UID: \"fe79c4ee-010a-45f9-9aac-6e56410702d7\") " Jul 22 19:25:09.031358 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031087 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:09.031358 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031288 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.031456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031387 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe79c4ee-010a-45f9-9aac-6e56410702d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:25:09.031525 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031505 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:09.031677 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031646 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:09.031822 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031794 2565 generic.go:356] "Generic (PLEG): container finished" podID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerID="b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c" exitCode=0 Jul 22 19:25:09.031881 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031854 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" Jul 22 19:25:09.031994 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.031953 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" event={"ID":"fe79c4ee-010a-45f9-9aac-6e56410702d7","Type":"ContainerDied","Data":"b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c"} Jul 22 19:25:09.032063 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.032009 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf" event={"ID":"fe79c4ee-010a-45f9-9aac-6e56410702d7","Type":"ContainerDied","Data":"168c91ed3ac949b54779ce5ab751a990d94e860a5535c43ab982e4894eeac017"} Jul 22 19:25:09.032063 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.032043 2565 scope.go:117] "RemoveContainer" containerID="b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c" Jul 22 19:25:09.032218 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.032194 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:09.033505 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.033479 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-kube-api-access-tcs8n" (OuterVolumeSpecName: "kube-api-access-tcs8n") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "kube-api-access-tcs8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:09.033593 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.033561 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:09.033725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.033690 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:25:09.033813 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.033782 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe79c4ee-010a-45f9-9aac-6e56410702d7" (UID: "fe79c4ee-010a-45f9-9aac-6e56410702d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:09.049987 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.049964 2565 scope.go:117] "RemoveContainer" containerID="3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66" Jul 22 19:25:09.057553 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.057533 2565 scope.go:117] "RemoveContainer" containerID="b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c" Jul 22 19:25:09.057916 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:25:09.057893 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c\": container with ID starting with b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c not found: ID does not exist" containerID="b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c" Jul 22 19:25:09.057983 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.057924 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c"} err="failed to get container status \"b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c\": rpc error: code = NotFound desc = could not find container \"b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c\": container with ID starting with b65fc97ea891ca2e521bb56adf83a45bce9e8af2a432ef72178bc35dd3e15b0c not found: ID does not exist" Jul 22 19:25:09.057983 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.057942 2565 scope.go:117] "RemoveContainer" containerID="3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66" Jul 22 19:25:09.058199 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:25:09.058182 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66\": container with ID starting with 3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66 not found: ID does not exist" containerID="3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66" Jul 22 19:25:09.058237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.058208 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66"} err="failed to get container status \"3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66\": rpc error: code = NotFound desc = could not find container \"3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66\": container with ID starting with 3e123d79f8aa99e256ba83852859910b857f3dbb648e96e23e5af3bf9ea3bd66 not found: ID does not exist" Jul 22 19:25:09.132250 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132214 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132250 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132244 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132250 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132254 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132261 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132277 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe79c4ee-010a-45f9-9aac-6e56410702d7-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132290 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132302 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcs8n\" (UniqueName: \"kubernetes.io/projected/fe79c4ee-010a-45f9-9aac-6e56410702d7-kube-api-access-tcs8n\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.132495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.132314 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe79c4ee-010a-45f9-9aac-6e56410702d7-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:09.354810 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.354775 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf"] Jul 22 19:25:09.365170 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.365141 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6058163f-gw-openshift-default-579bc8df7f-qmjwf"] Jul 22 19:25:09.813472 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:09.813427 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" path="/var/lib/kubelet/pods/fe79c4ee-010a-45f9-9aac-6e56410702d7/volumes" Jul 22 19:25:15.441395 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:15.441359 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:25:23.062808 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:23.062753 2565 patch_prober.go:28] interesting pod/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:25:23.063348 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:23.062831 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:25:24.108120 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:24.108087 2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerID="5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f" exitCode=0 Jul 22 19:25:24.108548 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:24.108158 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" event={"ID":"e8d85184-cf32-400a-9bf8-8d97fb4dd27f","Type":"ContainerDied","Data":"5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f"} Jul 22 19:25:24.108548 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:24.108531 2565 scope.go:117] "RemoveContainer" containerID="5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f" Jul 22 19:25:24.109659 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:24.109632 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:24.109727 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:24.109699 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:24.109765 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:24.109731 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:25.114244 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:25.114211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" event={"ID":"e8d85184-cf32-400a-9bf8-8d97fb4dd27f","Type":"ContainerStarted","Data":"4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab"} Jul 22 19:25:25.994244 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:25.994202 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:25.994433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:25.994408 2565 patch_prober.go:28] interesting pod/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 22 19:25:25.994484 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:25.994454 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 22 19:25:26.994742 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:26.994710 2565 patch_prober.go:28] interesting pod/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 22 19:25:26.995144 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:26.994766 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 22 19:25:27.994214 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:27.994173 2565 patch_prober.go:28] interesting pod/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 22 19:25:27.994398 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:27.994247 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 22 19:25:28.997200 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:28.997170 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:28.997579 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:28.997528 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:28.998150 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:28.998130 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:47.042147 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:47.042111 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:25:49.968408 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:49.968366 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v"] Jul 22 19:25:49.968909 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:49.968739 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" containerID="cri-o://4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab" gracePeriod=30 Jul 22 19:25:50.832993 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.832953 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c"] Jul 22 19:25:50.833448 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.833429 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" Jul 22 19:25:50.833499 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.833452 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" Jul 22 19:25:50.833499 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.833469 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" Jul 22 19:25:50.833499 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.833478 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" Jul 22 19:25:50.833596 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.833559 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" Jul 22 19:25:50.836549 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.836530 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.838950 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.838930 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-eaab3c34-openshift-default-dockercfg-r4kzc\"" Jul 22 19:25:50.846436 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.846411 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c"] Jul 22 19:25:50.900709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-credential-socket\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900704 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-certs\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900972 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-token\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900972 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900761 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttnqz\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-kube-api-access-ttnqz\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900972 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-podinfo\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900972 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istiod-ca-cert\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900972 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-data\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.900972 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.900937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-socket\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:50.901260 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:50.901006 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-envoy\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.001570 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-envoy\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-credential-socket\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-certs\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-token\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001681 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttnqz\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-kube-api-access-ttnqz\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001706 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-podinfo\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istiod-ca-cert\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001765 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-data\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.001799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-socket\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.002091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-certs\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002427 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.002147 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-socket\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002427 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.002257 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-data\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002427 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.002311 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-credential-socket\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.002582 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.002559 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istiod-ca-cert\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.003884 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.003866 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-envoy\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.004203 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.004181 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-podinfo\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.009698 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.009677 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-token\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.009908 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.009882 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttnqz\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-kube-api-access-ttnqz\") pod \"e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.148323 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.148222 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:51.284549 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.284524 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c"] Jul 22 19:25:51.286790 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:25:51.286761 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93b4ea6_ba78_493f_97a1_caa3d90e9b09.slice/crio-857b479c8b8e20249f6347a0c05417c21e937221bdddea65e3d90db75fb2ff67 WatchSource:0}: Error finding container 857b479c8b8e20249f6347a0c05417c21e937221bdddea65e3d90db75fb2ff67: Status 404 returned error can't find the container with id 857b479c8b8e20249f6347a0c05417c21e937221bdddea65e3d90db75fb2ff67 Jul 22 19:25:51.288960 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.288932 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:51.289038 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.288997 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:51.289038 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:51.289030 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:25:52.236461 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:52.236429 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" event={"ID":"a93b4ea6-ba78-493f-97a1-caa3d90e9b09","Type":"ContainerStarted","Data":"8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d"} Jul 22 19:25:52.236873 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:52.236480 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" event={"ID":"a93b4ea6-ba78-493f-97a1-caa3d90e9b09","Type":"ContainerStarted","Data":"857b479c8b8e20249f6347a0c05417c21e937221bdddea65e3d90db75fb2ff67"} Jul 22 19:25:52.256784 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:52.256732 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podStartSLOduration=2.256716333 podStartE2EDuration="2.256716333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:25:52.254718321 +0000 UTC m=+1752.981279423" watchObservedRunningTime="2026-07-22 19:25:52.256716333 +0000 UTC m=+1752.983277435" Jul 22 19:25:53.149352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:53.149316 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:53.153991 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:53.153968 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:53.241855 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:53.241828 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:53.242826 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:53.242809 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:25:55.225634 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.225591 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:55.236399 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236323 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-envoy\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236399 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236358 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istiod-ca-cert\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236643 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236409 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-credential-socket\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236643 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236441 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-socket\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236643 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236470 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-token\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236643 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236486 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-podinfo\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236643 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236511 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-certs\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236683 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22h6p\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-kube-api-access-22h6p\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236738 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-data\") pod \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\" (UID: \"e8d85184-cf32-400a-9bf8-8d97fb4dd27f\") " Jul 22 19:25:55.236897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236795 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.237052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236968 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:25:55.237052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236973 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.237052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.236990 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.237052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.237038 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.237335 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.237056 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.237704 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.237682 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.238829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.238806 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:25:55.238975 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.238861 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-kube-api-access-22h6p" (OuterVolumeSpecName: "kube-api-access-22h6p") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "kube-api-access-22h6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:55.238975 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.238939 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.238975 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.238951 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8d85184-cf32-400a-9bf8-8d97fb4dd27f" (UID: "e8d85184-cf32-400a-9bf8-8d97fb4dd27f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:55.255199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.255168 2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerID="4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab" exitCode=0 Jul 22 19:25:55.255346 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.255217 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" Jul 22 19:25:55.255346 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.255243 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" event={"ID":"e8d85184-cf32-400a-9bf8-8d97fb4dd27f","Type":"ContainerDied","Data":"4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab"} Jul 22 19:25:55.255346 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.255300 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v" event={"ID":"e8d85184-cf32-400a-9bf8-8d97fb4dd27f","Type":"ContainerDied","Data":"baeddeb6d352dda1e4e4b7971c6b5d9fae72892d324ea9715c78cdeaa0f23a6e"} Jul 22 19:25:55.255346 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.255324 2565 scope.go:117] "RemoveContainer" containerID="4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab" Jul 22 19:25:55.265029 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.265006 2565 scope.go:117] "RemoveContainer" containerID="5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f" Jul 22 19:25:55.273326 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.273305 2565 scope.go:117] "RemoveContainer" containerID="4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab" Jul 22 19:25:55.273694 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:25:55.273599 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab\": container with ID starting with 4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab not found: ID does not exist" containerID="4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab" Jul 22 19:25:55.273786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.273703 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab"} err="failed to get container status \"4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab\": rpc error: code = NotFound desc = could not find container \"4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab\": container with ID starting with 4076febd8207f5720bd4a6ef46931628b6f52fdb94b00a5ceb60f20c4416fdab not found: ID does not exist" Jul 22 19:25:55.273786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.273723 2565 scope.go:117] "RemoveContainer" containerID="5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f" Jul 22 19:25:55.274014 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:25:55.273990 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f\": container with ID starting with 5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f not found: ID does not exist" containerID="5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f" Jul 22 19:25:55.274054 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.274023 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f"} err="failed to get container status \"5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f\": rpc error: code = NotFound desc = could not find container \"5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f\": container with ID starting with 5c5da120739ba91860e9d51d0ad2a5acbfa01d08c3cce1a5153c2567f247118f not found: ID does not exist" Jul 22 19:25:55.277199 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.277175 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v"] Jul 22 19:25:55.281675 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.281653 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9c70fc9e-openshift-default-797d585b59-cwf8v"] Jul 22 19:25:55.338477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338437 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22h6p\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-kube-api-access-22h6p\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.338477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338468 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.338477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338479 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.338477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338487 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.338477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338495 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.338853 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338504 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.338853 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.338513 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8d85184-cf32-400a-9bf8-8d97fb4dd27f-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.813337 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:55.813306 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" path="/var/lib/kubelet/pods/e8d85184-cf32-400a-9bf8-8d97fb4dd27f/volumes" Jul 22 19:25:59.346120 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:25:59.346074 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:26:08.243909 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.243869 2565 patch_prober.go:28] interesting pod/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 22 19:26:08.244414 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.243936 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 22 19:26:08.320009 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.319974 2565 generic.go:356] "Generic (PLEG): container finished" podID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerID="8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d" exitCode=0 Jul 22 19:26:08.320171 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.320035 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" event={"ID":"a93b4ea6-ba78-493f-97a1-caa3d90e9b09","Type":"ContainerDied","Data":"8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d"} Jul 22 19:26:08.320407 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.320394 2565 scope.go:117] "RemoveContainer" containerID="8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d" Jul 22 19:26:08.321468 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.321438 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:26:08.321537 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.321520 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:26:08.321584 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:08.321572 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:26:09.328973 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:09.328931 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" event={"ID":"a93b4ea6-ba78-493f-97a1-caa3d90e9b09","Type":"ContainerStarted","Data":"28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691"} Jul 22 19:26:10.148536 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:10.148499 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:26:10.148768 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:10.148752 2565 patch_prober.go:28] interesting pod/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 22 19:26:10.148812 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:10.148795 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 22 19:26:11.149248 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:11.149211 2565 patch_prober.go:28] interesting pod/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 22 19:26:11.149676 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:11.149268 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 22 19:26:12.149255 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:12.149212 2565 patch_prober.go:28] interesting pod/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 22 19:26:12.149685 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:12.149272 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 22 19:26:13.149589 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:13.149545 2565 patch_prober.go:28] interesting pod/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 22 19:26:13.150027 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:13.149636 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 22 19:26:14.152508 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:14.152478 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:26:14.152906 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:14.152809 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:26:14.153467 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:14.153452 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:26:32.048221 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:32.048140 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:26:33.849624 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:33.849575 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c"] Jul 22 19:26:33.850038 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:33.849857 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" containerID="cri-o://28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691" gracePeriod=30 Jul 22 19:26:39.101957 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.101929 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:26:39.139357 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139274 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-token\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139357 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139322 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istiod-ca-cert\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139360 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-certs\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139390 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-envoy\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139441 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttnqz\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-kube-api-access-ttnqz\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139472 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-credential-socket\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139506 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-socket\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139546 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-podinfo\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139574 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-data\") pod \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\" (UID: \"a93b4ea6-ba78-493f-97a1-caa3d90e9b09\") " Jul 22 19:26:39.139793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139684 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:26:39.139793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139696 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:26:39.139994 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139906 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:26:39.139994 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139942 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.139994 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.139962 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.140133 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.140058 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:26:39.140965 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.140935 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-data" (OuterVolumeSpecName: "istio-data") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:26:39.141764 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.141735 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:26:39.141909 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.141888 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:26:39.142187 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.142161 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-kube-api-access-ttnqz" (OuterVolumeSpecName: "kube-api-access-ttnqz") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "kube-api-access-ttnqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:26:39.142187 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.142174 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-token" (OuterVolumeSpecName: "istio-token") pod "a93b4ea6-ba78-493f-97a1-caa3d90e9b09" (UID: "a93b4ea6-ba78-493f-97a1-caa3d90e9b09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:26:39.240350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240301 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.240350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240344 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.240350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240354 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttnqz\" (UniqueName: \"kubernetes.io/projected/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-kube-api-access-ttnqz\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.240350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240364 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.240350 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240373 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.240686 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240382 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.240686 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.240390 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93b4ea6-ba78-493f-97a1-caa3d90e9b09-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:26:39.462983 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.462953 2565 generic.go:356] "Generic (PLEG): container finished" podID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerID="28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691" exitCode=0 Jul 22 19:26:39.463191 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.463022 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" Jul 22 19:26:39.463191 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.463041 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" event={"ID":"a93b4ea6-ba78-493f-97a1-caa3d90e9b09","Type":"ContainerDied","Data":"28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691"} Jul 22 19:26:39.463191 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.463093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c" event={"ID":"a93b4ea6-ba78-493f-97a1-caa3d90e9b09","Type":"ContainerDied","Data":"857b479c8b8e20249f6347a0c05417c21e937221bdddea65e3d90db75fb2ff67"} Jul 22 19:26:39.463191 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.463117 2565 scope.go:117] "RemoveContainer" containerID="28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691" Jul 22 19:26:39.472317 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.472297 2565 scope.go:117] "RemoveContainer" containerID="8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d" Jul 22 19:26:39.481883 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.481860 2565 scope.go:117] "RemoveContainer" containerID="28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691" Jul 22 19:26:39.482200 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:26:39.482175 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691\": container with ID starting with 28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691 not found: ID does not exist" containerID="28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691" Jul 22 19:26:39.482328 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.482205 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691"} err="failed to get container status \"28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691\": rpc error: code = NotFound desc = could not find container \"28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691\": container with ID starting with 28856d9e9eee1d9989adda06de8cf19869f2db391f56754d4a62cbb3358c2691 not found: ID does not exist" Jul 22 19:26:39.482328 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.482225 2565 scope.go:117] "RemoveContainer" containerID="8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d" Jul 22 19:26:39.482562 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:26:39.482543 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d\": container with ID starting with 8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d not found: ID does not exist" containerID="8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d" Jul 22 19:26:39.482637 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.482568 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d"} err="failed to get container status \"8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d\": rpc error: code = NotFound desc = could not find container \"8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d\": container with ID starting with 8a0774000e9076f4dfe2ad3a98d27a73ee874480ba38e609e85fcfa991a2246d not found: ID does not exist" Jul 22 19:26:39.484806 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.484786 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c"] Jul 22 19:26:39.488159 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.488134 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-eaab3c34-openshift-default-77f755f9c5-l5c2c"] Jul 22 19:26:39.821651 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:26:39.821547 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" path="/var/lib/kubelet/pods/a93b4ea6-ba78-493f-97a1-caa3d90e9b09/volumes" Jul 22 19:28:00.041318 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:00.041278 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:28:09.542190 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:09.542107 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:28:12.057172 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057139 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp"] Jul 22 19:28:12.057535 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057456 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" Jul 22 19:28:12.057535 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057468 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" Jul 22 19:28:12.057535 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057480 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" Jul 22 19:28:12.057535 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057486 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" Jul 22 19:28:12.057535 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057494 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" Jul 22 19:28:12.057535 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057500 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" Jul 22 19:28:12.057754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057554 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" Jul 22 19:28:12.057754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057562 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" Jul 22 19:28:12.057754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057569 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" Jul 22 19:28:12.057754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057575 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe79c4ee-010a-45f9-9aac-6e56410702d7" containerName="istio-proxy" Jul 22 19:28:12.057754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.057583 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93b4ea6-ba78-493f-97a1-caa3d90e9b09" containerName="istio-proxy" Jul 22 19:28:12.060547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.060531 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.063373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.063349 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c31fc03e-openshift-default-dockercfg-6dx5p\"" Jul 22 19:28:12.085583 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.085549 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp"] Jul 22 19:28:12.158054 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158015 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istiod-ca-cert\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-data\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-socket\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158112 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-envoy\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158132 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-certs\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158154 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hqvh\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-kube-api-access-8hqvh\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-credential-socket\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-token\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.158386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.158282 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-podinfo\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.258929 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.258894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-data\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.258929 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.258935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-socket\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.258969 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-envoy\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.258990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-certs\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hqvh\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-kube-api-access-8hqvh\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-credential-socket\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-token\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259185 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-podinfo\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259486 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istiod-ca-cert\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259486 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259334 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-data\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259486 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259356 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-socket\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259486 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259450 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-credential-socket\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259670 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-certs\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.259939 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.259922 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istiod-ca-cert\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.261483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.261465 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-envoy\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.261743 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.261724 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-podinfo\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.266854 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.266833 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-token\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.267166 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.267144 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hqvh\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-kube-api-access-8hqvh\") pod \"e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.371232 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.371133 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:12.506413 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.506387 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp"] Jul 22 19:28:12.508658 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:28:12.508602 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64fbd42_a939_4c5e_954b_a5d5e581a7c0.slice/crio-b76374f3397f217c48c9b0b6c5eeffb63ab7489f3ce0c78dd7e230ad095332d5 WatchSource:0}: Error finding container b76374f3397f217c48c9b0b6c5eeffb63ab7489f3ce0c78dd7e230ad095332d5: Status 404 returned error can't find the container with id b76374f3397f217c48c9b0b6c5eeffb63ab7489f3ce0c78dd7e230ad095332d5 Jul 22 19:28:12.510753 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.510721 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:28:12.510836 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.510789 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:28:12.510836 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.510819 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:28:12.880940 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.880909 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" event={"ID":"a64fbd42-a939-4c5e-954b-a5d5e581a7c0","Type":"ContainerStarted","Data":"ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c"} Jul 22 19:28:12.881122 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.880958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" event={"ID":"a64fbd42-a939-4c5e-954b-a5d5e581a7c0","Type":"ContainerStarted","Data":"b76374f3397f217c48c9b0b6c5eeffb63ab7489f3ce0c78dd7e230ad095332d5"} Jul 22 19:28:12.902238 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:12.902184 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" podStartSLOduration=0.902167411 podStartE2EDuration="902.167411ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:28:12.901369194 +0000 UTC m=+1893.627930295" watchObservedRunningTime="2026-07-22 19:28:12.902167411 +0000 UTC m=+1893.628728514" Jul 22 19:28:13.371555 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:13.371517 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:13.376635 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:13.376584 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:13.885576 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:13.885542 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:13.886578 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:13.886561 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:18.640354 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:18.640316 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:28:26.945449 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:26.945323 2565 generic.go:356] "Generic (PLEG): container finished" podID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerID="ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c" exitCode=0 Jul 22 19:28:26.945449 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:26.945374 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" event={"ID":"a64fbd42-a939-4c5e-954b-a5d5e581a7c0","Type":"ContainerDied","Data":"ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c"} Jul 22 19:28:26.946049 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:26.945815 2565 scope.go:117] "RemoveContainer" containerID="ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c" Jul 22 19:28:26.946860 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:26.946829 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:28:26.946974 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:26.946916 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:28:26.947034 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:26.946969 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:28:27.952181 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:27.952142 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" event={"ID":"a64fbd42-a939-4c5e-954b-a5d5e581a7c0","Type":"ContainerStarted","Data":"1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049"} Jul 22 19:28:28.371779 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:28.371662 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:28.373197 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:28.373168 2565 patch_prober.go:28] interesting pod/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 22 19:28:28.373405 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:28.373374 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 22 19:28:28.885594 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:28.885550 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:29.371683 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:29.371648 2565 patch_prober.go:28] interesting pod/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 22 19:28:29.372278 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:29.372244 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 22 19:28:30.372094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:30.372046 2565 patch_prober.go:28] interesting pod/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 22 19:28:30.372641 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:30.372120 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 22 19:28:31.372442 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:31.372395 2565 patch_prober.go:28] interesting pod/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 22 19:28:31.373079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:31.372464 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 22 19:28:32.376038 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:32.376009 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:32.377034 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:32.377015 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:40.043011 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:40.042977 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:28:48.310925 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:48.310885 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp"] Jul 22 19:28:48.311423 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:48.311175 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" containerID="cri-o://1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049" gracePeriod=30 Jul 22 19:28:53.566793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.566768 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:53.720603 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720551 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-podinfo\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720603 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720642 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-certs\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720669 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-envoy\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720692 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-token\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720726 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-socket\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720775 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istiod-ca-cert\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720793 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-credential-socket\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720846 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hqvh\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-kube-api-access-8hqvh\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.720897 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.720876 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-data\") pod \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\" (UID: \"a64fbd42-a939-4c5e-954b-a5d5e581a7c0\") " Jul 22 19:28:53.721242 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721018 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:53.721242 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721062 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:53.721242 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721165 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.721242 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721166 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:53.721242 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721188 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.721494 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721289 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:28:53.721841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.721817 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:53.723008 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.722980 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:28:53.723119 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.723075 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:53.723268 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.723238 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:28:53.723341 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.723326 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-kube-api-access-8hqvh" (OuterVolumeSpecName: "kube-api-access-8hqvh") pod "a64fbd42-a939-4c5e-954b-a5d5e581a7c0" (UID: "a64fbd42-a939-4c5e-954b-a5d5e581a7c0"). InnerVolumeSpecName "kube-api-access-8hqvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:28:53.822527 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822504 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.822633 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822528 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.822633 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822540 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hqvh\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-kube-api-access-8hqvh\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.822633 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822549 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.822633 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822557 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.822633 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822565 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:53.822633 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:53.822573 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64fbd42-a939-4c5e-954b-a5d5e581a7c0-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:28:54.066776 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.066684 2565 generic.go:356] "Generic (PLEG): container finished" podID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerID="1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049" exitCode=0 Jul 22 19:28:54.066776 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.066746 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" event={"ID":"a64fbd42-a939-4c5e-954b-a5d5e581a7c0","Type":"ContainerDied","Data":"1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049"} Jul 22 19:28:54.066948 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.066791 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" event={"ID":"a64fbd42-a939-4c5e-954b-a5d5e581a7c0","Type":"ContainerDied","Data":"b76374f3397f217c48c9b0b6c5eeffb63ab7489f3ce0c78dd7e230ad095332d5"} Jul 22 19:28:54.066948 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.066813 2565 scope.go:117] "RemoveContainer" containerID="1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049" Jul 22 19:28:54.066948 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.066753 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp" Jul 22 19:28:54.077826 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.077805 2565 scope.go:117] "RemoveContainer" containerID="ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c" Jul 22 19:28:54.085893 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.085762 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp"] Jul 22 19:28:54.085969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.085911 2565 scope.go:117] "RemoveContainer" containerID="1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049" Jul 22 19:28:54.086226 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:28:54.086207 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049\": container with ID starting with 1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049 not found: ID does not exist" containerID="1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049" Jul 22 19:28:54.086296 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.086232 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049"} err="failed to get container status \"1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049\": rpc error: code = NotFound desc = could not find container \"1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049\": container with ID starting with 1dc0fb14929722d6a46e434071ac9fc16d84fc902336b822fd49ae0a5b13b049 not found: ID does not exist" Jul 22 19:28:54.086296 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.086250 2565 scope.go:117] "RemoveContainer" containerID="ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c" Jul 22 19:28:54.086483 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:28:54.086465 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c\": container with ID starting with ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c not found: ID does not exist" containerID="ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c" Jul 22 19:28:54.086521 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.086490 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c"} err="failed to get container status \"ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c\": rpc error: code = NotFound desc = could not find container \"ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c\": container with ID starting with ac3fc571b8a0257d0a407b3022d4041a6ea4dc5240057a7908ee0078e32c970c not found: ID does not exist" Jul 22 19:28:54.092457 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:54.092432 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c31fc03e-openshift-default-6fd9f46d47-5xfhp"] Jul 22 19:28:55.813755 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:28:55.813724 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" path="/var/lib/kubelet/pods/a64fbd42-a939-4c5e-954b-a5d5e581a7c0/volumes" Jul 22 19:29:08.021034 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.020998 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v"] Jul 22 19:29:08.021681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.021454 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" Jul 22 19:29:08.021681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.021472 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" Jul 22 19:29:08.021681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.021503 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" Jul 22 19:29:08.021681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.021513 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d85184-cf32-400a-9bf8-8d97fb4dd27f" containerName="istio-proxy" Jul 22 19:29:08.021681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.021600 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" Jul 22 19:29:08.026046 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.026021 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.028789 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.028763 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7a0d319a-openshift-default-dockercfg-mcfct\"" Jul 22 19:29:08.038572 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.038545 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v"] Jul 22 19:29:08.044288 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-envoy\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044427 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-credential-socket\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044427 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-socket\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044517 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-certs\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044517 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044454 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-data\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044517 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044501 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-token\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044663 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sv2r\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-kube-api-access-4sv2r\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044663 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-podinfo\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.044736 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.044660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb6a4b-61a8-4aff-89df-79646efc4083-istiod-ca-cert\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.145681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-credential-socket\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.145887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-socket\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.145887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-certs\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.145887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-data\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.145887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-token\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.145887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sv2r\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-kube-api-access-4sv2r\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-podinfo\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb6a4b-61a8-4aff-89df-79646efc4083-istiod-ca-cert\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.145978 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-envoy\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.146101 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-credential-socket\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.146130 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-certs\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146542 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.146185 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-socket\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.146718 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.146693 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-data\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.147001 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.146830 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb6a4b-61a8-4aff-89df-79646efc4083-istiod-ca-cert\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.148721 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.148698 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-envoy\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.148923 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.148907 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-podinfo\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.153676 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.153651 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-token\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.153782 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.153680 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sv2r\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-kube-api-access-4sv2r\") pod \"e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.336108 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.336020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:08.469686 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.469662 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v"] Jul 22 19:29:08.471468 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:29:08.471441 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37cb6a4b_61a8_4aff_89df_79646efc4083.slice/crio-5d649ee088b2361340b807c139b877c061de1ea78be98ffd9f1d87c74fa46853 WatchSource:0}: Error finding container 5d649ee088b2361340b807c139b877c061de1ea78be98ffd9f1d87c74fa46853: Status 404 returned error can't find the container with id 5d649ee088b2361340b807c139b877c061de1ea78be98ffd9f1d87c74fa46853 Jul 22 19:29:08.473690 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.473661 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:29:08.473756 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.473726 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:29:08.473792 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:08.473757 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:29:09.133935 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:09.133899 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" event={"ID":"37cb6a4b-61a8-4aff-89df-79646efc4083","Type":"ContainerStarted","Data":"5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277"} Jul 22 19:29:09.134375 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:09.133958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" event={"ID":"37cb6a4b-61a8-4aff-89df-79646efc4083","Type":"ContainerStarted","Data":"5d649ee088b2361340b807c139b877c061de1ea78be98ffd9f1d87c74fa46853"} Jul 22 19:29:09.156048 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:09.156004 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podStartSLOduration=2.155986284 podStartE2EDuration="2.155986284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:29:09.154740842 +0000 UTC m=+1949.881301941" watchObservedRunningTime="2026-07-22 19:29:09.155986284 +0000 UTC m=+1949.882547386" Jul 22 19:29:09.336904 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:09.336865 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:09.342162 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:09.342135 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:10.138512 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:10.138479 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:10.139420 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:10.139401 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:17.745619 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:17.745568 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:29:25.359756 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:25.359404 2565 patch_prober.go:28] interesting pod/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:41538->10.134.0.38:15021: read: connection reset by peer" start-of-body= Jul 22 19:29:25.359756 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:25.359487 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:41538->10.134.0.38:15021: read: connection reset by peer" Jul 22 19:29:26.215195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:26.215160 2565 generic.go:356] "Generic (PLEG): container finished" podID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerID="5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277" exitCode=0 Jul 22 19:29:26.215397 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:26.215206 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" event={"ID":"37cb6a4b-61a8-4aff-89df-79646efc4083","Type":"ContainerDied","Data":"5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277"} Jul 22 19:29:26.215597 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:26.215583 2565 scope.go:117] "RemoveContainer" containerID="5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277" Jul 22 19:29:26.216567 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:26.216538 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:29:26.216653 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:26.216620 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:29:26.216689 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:26.216651 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:29:27.221810 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:27.221770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" event={"ID":"37cb6a4b-61a8-4aff-89df-79646efc4083","Type":"ContainerStarted","Data":"fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d"} Jul 22 19:29:27.336292 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:27.336255 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:27.336508 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:27.336489 2565 patch_prober.go:28] interesting pod/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:29:27.336553 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:27.336531 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:29:28.336774 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:28.336735 2565 patch_prober.go:28] interesting pod/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:29:28.337164 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:28.336791 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:29:29.337287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:29.337247 2565 patch_prober.go:28] interesting pod/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:29:29.337733 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:29.337323 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:29:30.337381 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:30.337344 2565 patch_prober.go:28] interesting pod/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:29:30.337819 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:30.337412 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:29:31.340221 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:31.340141 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:31.340683 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:31.340436 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:31.341183 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:31.341168 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:29:38.048514 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:38.048475 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:29:50.143686 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:29:50.143654 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:30:08.142887 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:30:08.142849 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:30:17.638803 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:30:17.638752 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:31:04.066480 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:04.066400 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v"] Jul 22 19:31:04.067042 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:04.066637 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" containerID="cri-o://fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d" gracePeriod=30 Jul 22 19:31:07.951451 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:07.951414 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:31:09.319314 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.319290 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:31:09.416235 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416201 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-data\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416245 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-certs\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416300 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-token\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416316 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-podinfo\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416337 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb6a4b-61a8-4aff-89df-79646efc4083-istiod-ca-cert\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416411 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-socket\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416437 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sv2r\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-kube-api-access-4sv2r\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416467 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-envoy\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.416511 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416504 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-credential-socket\") pod \"37cb6a4b-61a8-4aff-89df-79646efc4083\" (UID: \"37cb6a4b-61a8-4aff-89df-79646efc4083\") " Jul 22 19:31:09.417012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416694 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:09.417012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416791 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.417012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416842 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37cb6a4b-61a8-4aff-89df-79646efc4083-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:31:09.417012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416877 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:09.417012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.416963 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:09.417280 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.417137 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-data" (OuterVolumeSpecName: "istio-data") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:09.418933 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.418897 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:31:09.419064 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.418974 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-kube-api-access-4sv2r" (OuterVolumeSpecName: "kube-api-access-4sv2r") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "kube-api-access-4sv2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:31:09.419064 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.419040 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-token" (OuterVolumeSpecName: "istio-token") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:31:09.419195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.419033 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37cb6a4b-61a8-4aff-89df-79646efc4083" (UID: "37cb6a4b-61a8-4aff-89df-79646efc4083"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:09.517754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517719 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517751 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sv2r\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-kube-api-access-4sv2r\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517762 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517771 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517779 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517787 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517795 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb6a4b-61a8-4aff-89df-79646efc4083-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.517995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.517803 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb6a4b-61a8-4aff-89df-79646efc4083-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:31:09.667076 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.666978 2565 generic.go:356] "Generic (PLEG): container finished" podID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerID="fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d" exitCode=0 Jul 22 19:31:09.667076 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.667027 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" Jul 22 19:31:09.667076 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.667031 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" event={"ID":"37cb6a4b-61a8-4aff-89df-79646efc4083","Type":"ContainerDied","Data":"fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d"} Jul 22 19:31:09.667364 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.667111 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v" event={"ID":"37cb6a4b-61a8-4aff-89df-79646efc4083","Type":"ContainerDied","Data":"5d649ee088b2361340b807c139b877c061de1ea78be98ffd9f1d87c74fa46853"} Jul 22 19:31:09.667364 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.667137 2565 scope.go:117] "RemoveContainer" containerID="fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d" Jul 22 19:31:09.678011 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.677990 2565 scope.go:117] "RemoveContainer" containerID="5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277" Jul 22 19:31:09.686557 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.686527 2565 scope.go:117] "RemoveContainer" containerID="fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d" Jul 22 19:31:09.686861 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:31:09.686841 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d\": container with ID starting with fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d not found: ID does not exist" containerID="fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d" Jul 22 19:31:09.686951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.686875 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d"} err="failed to get container status \"fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d\": rpc error: code = NotFound desc = could not find container \"fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d\": container with ID starting with fac9b6717ee82776223520fb2c506e27629e1851d851722edc632f7e5e6eff7d not found: ID does not exist" Jul 22 19:31:09.686951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.686901 2565 scope.go:117] "RemoveContainer" containerID="5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277" Jul 22 19:31:09.687199 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:31:09.687165 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277\": container with ID starting with 5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277 not found: ID does not exist" containerID="5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277" Jul 22 19:31:09.687248 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.687212 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277"} err="failed to get container status \"5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277\": rpc error: code = NotFound desc = could not find container \"5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277\": container with ID starting with 5f56de4f1c5da844cac13dbf9379016db6e7e6f3fc37cc28ffafdadd1a912277 not found: ID does not exist" Jul 22 19:31:09.691344 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.691315 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v"] Jul 22 19:31:09.695388 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.695359 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7a0d319a-openshift-default-7c7bbd4b87-9cb6v"] Jul 22 19:31:09.814779 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:09.814742 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" path="/var/lib/kubelet/pods/37cb6a4b-61a8-4aff-89df-79646efc4083/volumes" Jul 22 19:31:19.742532 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:19.742495 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:31:58.918228 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918185 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m"] Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918494 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918505 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918522 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918527 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918582 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918589 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a64fbd42-a939-4c5e-954b-a5d5e581a7c0" containerName="istio-proxy" Jul 22 19:31:58.918750 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.918596 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" Jul 22 19:31:58.921622 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.921581 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:58.924144 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.924122 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-735d50-openshift-default-dockercfg-lpn4b\"" Jul 22 19:31:58.933661 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:58.933634 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m"] Jul 22 19:31:59.064846 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064806 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-token\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.064846 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-envoy\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9txhs\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-kube-api-access-9txhs\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064902 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-credential-socket\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-podinfo\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064939 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-certs\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.064962 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fcd5a0f-086a-4958-af2e-e6981691c87b-istiod-ca-cert\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.065059 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-socket\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.065440 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.065119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-data\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166070 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9txhs\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-kube-api-access-9txhs\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166070 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-credential-socket\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166345 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166093 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-podinfo\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166345 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166116 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-certs\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166345 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fcd5a0f-086a-4958-af2e-e6981691c87b-istiod-ca-cert\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166345 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166198 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-socket\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166345 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-data\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166345 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-token\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166690 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166366 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-envoy\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166690 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166559 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-certs\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166690 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166636 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-data\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166852 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166682 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-socket\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.166909 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.166887 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-credential-socket\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.167027 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.167007 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fcd5a0f-086a-4958-af2e-e6981691c87b-istiod-ca-cert\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.169112 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.169054 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-envoy\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.169218 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.169174 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-podinfo\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.174731 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.174707 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-token\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.174902 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.174880 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9txhs\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-kube-api-access-9txhs\") pod \"e2e-inf-a-735d50-openshift-default-789875c949-fj69m\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.232485 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.232446 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:31:59.371217 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.371019 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m"] Jul 22 19:31:59.373970 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:31:59.373942 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcd5a0f_086a_4958_af2e_e6981691c87b.slice/crio-78a833d55502960bf9fbeb082109feebe476c2c9c6bd4c789f6f05d186604bd7 WatchSource:0}: Error finding container 78a833d55502960bf9fbeb082109feebe476c2c9c6bd4c789f6f05d186604bd7: Status 404 returned error can't find the container with id 78a833d55502960bf9fbeb082109feebe476c2c9c6bd4c789f6f05d186604bd7 Jul 22 19:31:59.376226 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.376190 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:31:59.376314 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.376261 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:31:59.376314 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.376291 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:31:59.882410 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.882314 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" event={"ID":"1fcd5a0f-086a-4958-af2e-e6981691c87b","Type":"ContainerStarted","Data":"54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed"} Jul 22 19:31:59.882410 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.882376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" event={"ID":"1fcd5a0f-086a-4958-af2e-e6981691c87b","Type":"ContainerStarted","Data":"78a833d55502960bf9fbeb082109feebe476c2c9c6bd4c789f6f05d186604bd7"} Jul 22 19:31:59.904311 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:31:59.904261 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podStartSLOduration=1.904240337 podStartE2EDuration="1.904240337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:59.902575444 +0000 UTC m=+2120.629136553" watchObservedRunningTime="2026-07-22 19:31:59.904240337 +0000 UTC m=+2120.630801439" Jul 22 19:32:00.233053 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:00.233015 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:00.238034 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:00.238007 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:00.886195 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:00.886154 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:00.887298 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:00.887274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:06.976671 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:06.976637 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd"] Jul 22 19:32:06.977239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:06.977019 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" Jul 22 19:32:06.977239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:06.977034 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="37cb6a4b-61a8-4aff-89df-79646efc4083" containerName="istio-proxy" Jul 22 19:32:06.980436 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:06.980415 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:06.983044 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:06.983026 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-dc2828-openshift-default-dockercfg-7rq45\"" Jul 22 19:32:06.993007 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:06.992979 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd"] Jul 22 19:32:07.136478 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-certs\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136717 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136490 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-istio-token\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136717 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-envoy\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136717 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136571 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbwxd\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-kube-api-access-rbwxd\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136717 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136677 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-socket\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-credential-socket\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136760 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062731b1-13b2-48b2-8043-97a3476c19e7-istio-podinfo\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136798 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-data\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.136969 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.136828 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062731b1-13b2-48b2-8043-97a3476c19e7-istiod-ca-cert\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.237890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.237793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-credential-socket\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.237890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.237838 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062731b1-13b2-48b2-8043-97a3476c19e7-istio-podinfo\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.237890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.237875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-data\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.237903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062731b1-13b2-48b2-8043-97a3476c19e7-istiod-ca-cert\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.237954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-certs\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.237995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-istio-token\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-envoy\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238239 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbwxd\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-kube-api-access-rbwxd\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238503 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-socket\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238712 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238688 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-certs\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238861 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238840 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-credential-socket\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.238943 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238863 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-socket\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.239010 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.238972 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-data\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.239075 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.239053 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062731b1-13b2-48b2-8043-97a3476c19e7-istiod-ca-cert\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.240476 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.240451 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-envoy\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.240774 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.240757 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062731b1-13b2-48b2-8043-97a3476c19e7-istio-podinfo\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.246294 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.246263 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-istio-token\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.246416 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.246400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbwxd\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-kube-api-access-rbwxd\") pod \"e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.291870 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.291838 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:07.432947 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.432918 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd"] Jul 22 19:32:07.434543 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:32:07.434516 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062731b1_13b2_48b2_8043_97a3476c19e7.slice/crio-9121e937a085d51b1f9d9e67e91cb407bd198d52ceec01bfe99c6fb53f4e61c2 WatchSource:0}: Error finding container 9121e937a085d51b1f9d9e67e91cb407bd198d52ceec01bfe99c6fb53f4e61c2: Status 404 returned error can't find the container with id 9121e937a085d51b1f9d9e67e91cb407bd198d52ceec01bfe99c6fb53f4e61c2 Jul 22 19:32:07.437045 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.437012 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:07.437116 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.437081 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:07.437116 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.437110 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:07.918050 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.918008 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" event={"ID":"062731b1-13b2-48b2-8043-97a3476c19e7","Type":"ContainerStarted","Data":"1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49"} Jul 22 19:32:07.918237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.918063 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" event={"ID":"062731b1-13b2-48b2-8043-97a3476c19e7","Type":"ContainerStarted","Data":"9121e937a085d51b1f9d9e67e91cb407bd198d52ceec01bfe99c6fb53f4e61c2"} Jul 22 19:32:07.944203 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:07.944150 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" podStartSLOduration=1.9441332679999999 podStartE2EDuration="1.944133268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:32:07.941978194 +0000 UTC m=+2128.668539296" watchObservedRunningTime="2026-07-22 19:32:07.944133268 +0000 UTC m=+2128.670694419" Jul 22 19:32:08.116587 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:08.116550 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:32:08.292330 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:08.292235 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:08.297702 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:08.297678 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:08.923113 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:08.923084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:08.924040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:08.924017 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:16.662447 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.662414 2565 patch_prober.go:28] interesting pod/e2e-inf-a-735d50-openshift-default-789875c949-fj69m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:55946->10.134.0.39:15021: read: connection reset by peer" start-of-body= Jul 22 19:32:16.663321 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.662999 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:55946->10.134.0.39:15021: read: connection reset by peer" Jul 22 19:32:16.960575 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.960498 2565 generic.go:356] "Generic (PLEG): container finished" podID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerID="54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed" exitCode=0 Jul 22 19:32:16.960575 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.960566 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" event={"ID":"1fcd5a0f-086a-4958-af2e-e6981691c87b","Type":"ContainerDied","Data":"54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed"} Jul 22 19:32:16.960996 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.960978 2565 scope.go:117] "RemoveContainer" containerID="54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed" Jul 22 19:32:16.962379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.962016 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:16.962379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.962109 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:16.962379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:16.962158 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:17.974102 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:17.973175 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" event={"ID":"1fcd5a0f-086a-4958-af2e-e6981691c87b","Type":"ContainerStarted","Data":"5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481"} Jul 22 19:32:18.233479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:18.233374 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:18.233780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:18.233757 2565 patch_prober.go:28] interesting pod/e2e-inf-a-735d50-openshift-default-789875c949-fj69m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 22 19:32:18.233893 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:18.233810 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 22 19:32:19.233508 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:19.233470 2565 patch_prober.go:28] interesting pod/e2e-inf-a-735d50-openshift-default-789875c949-fj69m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 22 19:32:19.233918 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:19.233530 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 22 19:32:20.233840 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:20.233796 2565 patch_prober.go:28] interesting pod/e2e-inf-a-735d50-openshift-default-789875c949-fj69m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 22 19:32:20.234244 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:20.233887 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 22 19:32:20.346733 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:20.346694 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:32:21.233211 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:21.233173 2565 patch_prober.go:28] interesting pod/e2e-inf-a-735d50-openshift-default-789875c949-fj69m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 22 19:32:21.233390 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:21.233232 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 22 19:32:22.235953 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:22.235924 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:22.236371 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:22.236250 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:22.236871 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:22.236853 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:32:23.001715 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.001681 2565 generic.go:356] "Generic (PLEG): container finished" podID="062731b1-13b2-48b2-8043-97a3476c19e7" containerID="1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49" exitCode=0 Jul 22 19:32:23.001904 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.001765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" event={"ID":"062731b1-13b2-48b2-8043-97a3476c19e7","Type":"ContainerDied","Data":"1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49"} Jul 22 19:32:23.002162 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.002145 2565 scope.go:117] "RemoveContainer" containerID="1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49" Jul 22 19:32:23.003099 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.003068 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:23.003204 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.003151 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:23.003275 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.003211 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:32:23.923880 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:23.923841 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:24.007793 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:24.007761 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" event={"ID":"062731b1-13b2-48b2-8043-97a3476c19e7","Type":"ContainerStarted","Data":"75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab"} Jul 22 19:32:24.293153 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:24.293054 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:24.294224 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:24.294189 2565 patch_prober.go:28] interesting pod/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 22 19:32:24.294356 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:24.294255 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 22 19:32:25.293363 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:25.293322 2565 patch_prober.go:28] interesting pod/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 22 19:32:25.293803 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:25.293397 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 22 19:32:26.293153 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:26.293110 2565 patch_prober.go:28] interesting pod/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 22 19:32:26.293342 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:26.293173 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 22 19:32:27.293071 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:27.293031 2565 patch_prober.go:28] interesting pod/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 22 19:32:27.293472 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:27.293096 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 22 19:32:28.297000 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:28.296966 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:28.297482 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:28.297374 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:28.298106 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:28.298084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:32:37.450446 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:37.450361 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:32:51.744079 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:32:51.744041 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:33:00.139214 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.139177 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t"] Jul 22 19:33:00.141748 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.141730 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.144159 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.144126 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-dc2828\"/\"test-model-dc2828-kserve-self-signed-certs\"" Jul 22 19:33:00.145194 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.145175 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-dc2828\"/\"kube-root-ca.crt\"" Jul 22 19:33:00.145319 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.145175 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-dc2828\"/\"openshift-service-ca.crt\"" Jul 22 19:33:00.145319 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.145293 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-dc2828\"/\"default-dockercfg-v9cdk\"" Jul 22 19:33:00.155398 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.155373 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t"] Jul 22 19:33:00.231025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.230989 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-home\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.231025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.231031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tmp-dir\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.231273 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.231120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-dshm\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.231273 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.231175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-model-cache\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.231273 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.231223 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tls-certs\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.231273 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.231257 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kt5r\" (UniqueName: \"kubernetes.io/projected/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-kube-api-access-6kt5r\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.332730 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.332692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-dshm\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.332951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.332753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-model-cache\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.332951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.332793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tls-certs\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.332951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.332828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kt5r\" (UniqueName: \"kubernetes.io/projected/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-kube-api-access-6kt5r\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.332951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.332887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-home\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.332951 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.332911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tmp-dir\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.333298 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.333275 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-model-cache\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.333298 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.333287 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-home\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.333438 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.333322 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tmp-dir\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.335171 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.335143 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-dshm\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.335432 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.335405 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tls-certs\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.341087 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.341058 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kt5r\" (UniqueName: \"kubernetes.io/projected/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-kube-api-access-6kt5r\") pod \"test-model-dc2828-kserve-744ff6fc87-mtc7t\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.453342 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.453293 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:00.590879 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:00.590856 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t"] Jul 22 19:33:00.593015 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:33:00.592990 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3da1b12c_0732_4006_b8b2_6c8c7ac02dc4.slice/crio-f410d928a44908a836eb9363b73722a7da6d726ce789cb0e2857964825b80912 WatchSource:0}: Error finding container f410d928a44908a836eb9363b73722a7da6d726ce789cb0e2857964825b80912: Status 404 returned error can't find the container with id f410d928a44908a836eb9363b73722a7da6d726ce789cb0e2857964825b80912 Jul 22 19:33:01.170280 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:01.170242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" event={"ID":"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4","Type":"ContainerStarted","Data":"447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d"} Jul 22 19:33:01.170280 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:01.170283 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" event={"ID":"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4","Type":"ContainerStarted","Data":"f410d928a44908a836eb9363b73722a7da6d726ce789cb0e2857964825b80912"} Jul 22 19:33:01.188497 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:01.188443 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" podStartSLOduration=1.188418728 podStartE2EDuration="1.188418728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:33:01.186043449 +0000 UTC m=+2181.912604553" watchObservedRunningTime="2026-07-22 19:33:01.188418728 +0000 UTC m=+2181.914979831" Jul 22 19:33:06.445431 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:06.445394 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:33:10.453884 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:10.453840 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:10.453884 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:10.453895 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:10.466531 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:10.466499 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:11.227037 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:11.227007 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:33:50.956731 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:50.956696 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t"] Jul 22 19:33:54.846499 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:33:54.846459 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:34:05.977262 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:05.977154 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" podUID="3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" containerName="main" containerID="cri-o://447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d" gracePeriod=45 Jul 22 19:34:06.229843 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.229777 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:34:06.335291 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335255 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-model-cache\") pod \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " Jul 22 19:34:06.335479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335325 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tmp-dir\") pod \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " Jul 22 19:34:06.335479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335349 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tls-certs\") pod \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " Jul 22 19:34:06.335479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335369 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kt5r\" (UniqueName: \"kubernetes.io/projected/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-kube-api-access-6kt5r\") pod \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " Jul 22 19:34:06.335479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335404 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-home\") pod \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " Jul 22 19:34:06.335479 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335426 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-dshm\") pod \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\" (UID: \"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4\") " Jul 22 19:34:06.335786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335549 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-model-cache" (OuterVolumeSpecName: "model-cache") pod "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" (UID: "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:06.335786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335574 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" (UID: "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:06.335786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335679 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-home" (OuterVolumeSpecName: "home") pod "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" (UID: "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:06.335786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335774 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-model-cache\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:06.335786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335787 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tmp-dir\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:06.335964 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.335795 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-home\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:06.337725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.337699 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-dshm" (OuterVolumeSpecName: "dshm") pod "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" (UID: "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:06.337839 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.337725 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" (UID: "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:34:06.337839 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.337758 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-kube-api-access-6kt5r" (OuterVolumeSpecName: "kube-api-access-6kt5r") pod "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" (UID: "3da1b12c-0732-4006-b8b2-6c8c7ac02dc4"). InnerVolumeSpecName "kube-api-access-6kt5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:06.436938 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.436892 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-tls-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:06.436938 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.436931 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kt5r\" (UniqueName: \"kubernetes.io/projected/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-kube-api-access-6kt5r\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:06.436938 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.436942 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4-dshm\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:06.465893 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.465858 2565 generic.go:356] "Generic (PLEG): container finished" podID="3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" containerID="447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d" exitCode=0 Jul 22 19:34:06.465893 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.465897 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" event={"ID":"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4","Type":"ContainerDied","Data":"447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d"} Jul 22 19:34:06.466117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.465904 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" Jul 22 19:34:06.466117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.465925 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t" event={"ID":"3da1b12c-0732-4006-b8b2-6c8c7ac02dc4","Type":"ContainerDied","Data":"f410d928a44908a836eb9363b73722a7da6d726ce789cb0e2857964825b80912"} Jul 22 19:34:06.466117 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.465942 2565 scope.go:117] "RemoveContainer" containerID="447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d" Jul 22 19:34:06.474346 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.474331 2565 scope.go:117] "RemoveContainer" containerID="447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d" Jul 22 19:34:06.474587 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:34:06.474571 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d\": container with ID starting with 447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d not found: ID does not exist" containerID="447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d" Jul 22 19:34:06.474701 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.474592 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d"} err="failed to get container status \"447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d\": rpc error: code = NotFound desc = could not find container \"447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d\": container with ID starting with 447946c4d177f0fcf584961e244c04de3d3b4574ba2c887fe048c79be97dc25d not found: ID does not exist" Jul 22 19:34:06.486457 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.486394 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t"] Jul 22 19:34:06.491709 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:06.491683 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-dc2828/test-model-dc2828-kserve-744ff6fc87-mtc7t"] Jul 22 19:34:07.813315 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:07.813283 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" path="/var/lib/kubelet/pods/3da1b12c-0732-4006-b8b2-6c8c7ac02dc4/volumes" Jul 22 19:34:10.242945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:10.242911 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:34:12.486354 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:12.486318 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m"] Jul 22 19:34:12.486874 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:12.486579 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" containerID="cri-o://5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481" gracePeriod=30 Jul 22 19:34:17.740162 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.740135 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:34:17.837495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837400 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9txhs\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-kube-api-access-9txhs\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837439 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fcd5a0f-086a-4958-af2e-e6981691c87b-istiod-ca-cert\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837457 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-data\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837495 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837489 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-socket\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837572 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-certs\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837597 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-credential-socket\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837650 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-podinfo\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837709 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-token\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.837872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837746 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-envoy\") pod \"1fcd5a0f-086a-4958-af2e-e6981691c87b\" (UID: \"1fcd5a0f-086a-4958-af2e-e6981691c87b\") " Jul 22 19:34:17.838118 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837900 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:17.838118 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837942 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:17.838118 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.837974 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcd5a0f-086a-4958-af2e-e6981691c87b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:34:17.838118 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.838096 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fcd5a0f-086a-4958-af2e-e6981691c87b-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.838118 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.838115 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.838363 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.838127 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.838363 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.838202 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:17.838363 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.838326 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:17.839752 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.839720 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-kube-api-access-9txhs" (OuterVolumeSpecName: "kube-api-access-9txhs") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "kube-api-access-9txhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:17.839863 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.839792 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:34:17.839915 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.839855 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:17.840046 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.840025 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fcd5a0f-086a-4958-af2e-e6981691c87b" (UID: "1fcd5a0f-086a-4958-af2e-e6981691c87b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:17.939408 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.939369 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.939408 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.939399 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.939408 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.939409 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9txhs\" (UniqueName: \"kubernetes.io/projected/1fcd5a0f-086a-4958-af2e-e6981691c87b-kube-api-access-9txhs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.939408 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.939418 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.939713 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.939427 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fcd5a0f-086a-4958-af2e-e6981691c87b-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:17.939713 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:17.939436 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fcd5a0f-086a-4958-af2e-e6981691c87b-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:18.521147 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.521111 2565 generic.go:356] "Generic (PLEG): container finished" podID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerID="5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481" exitCode=0 Jul 22 19:34:18.521352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.521158 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" Jul 22 19:34:18.521352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.521195 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" event={"ID":"1fcd5a0f-086a-4958-af2e-e6981691c87b","Type":"ContainerDied","Data":"5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481"} Jul 22 19:34:18.521352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.521261 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m" event={"ID":"1fcd5a0f-086a-4958-af2e-e6981691c87b","Type":"ContainerDied","Data":"78a833d55502960bf9fbeb082109feebe476c2c9c6bd4c789f6f05d186604bd7"} Jul 22 19:34:18.521352 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.521292 2565 scope.go:117] "RemoveContainer" containerID="5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481" Jul 22 19:34:18.529841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.529825 2565 scope.go:117] "RemoveContainer" containerID="54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed" Jul 22 19:34:18.538668 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.538649 2565 scope.go:117] "RemoveContainer" containerID="5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481" Jul 22 19:34:18.538951 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:34:18.538933 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481\": container with ID starting with 5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481 not found: ID does not exist" containerID="5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481" Jul 22 19:34:18.539025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.538964 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481"} err="failed to get container status \"5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481\": rpc error: code = NotFound desc = could not find container \"5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481\": container with ID starting with 5adda8aa6f6c99022b2df880c7a48a5d0e931633793b07fed3eeec055d056481 not found: ID does not exist" Jul 22 19:34:18.539025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.538988 2565 scope.go:117] "RemoveContainer" containerID="54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed" Jul 22 19:34:18.539228 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:34:18.539209 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed\": container with ID starting with 54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed not found: ID does not exist" containerID="54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed" Jul 22 19:34:18.539269 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.539236 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed"} err="failed to get container status \"54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed\": rpc error: code = NotFound desc = could not find container \"54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed\": container with ID starting with 54528a164413fc6b40d9537fec2d8acc1afd7d5843563bbd2e8c2de6190f03ed not found: ID does not exist" Jul 22 19:34:18.542205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.542181 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m"] Jul 22 19:34:18.546171 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:18.546149 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-735d50-openshift-default-789875c949-fj69m"] Jul 22 19:34:19.819386 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:19.819341 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" path="/var/lib/kubelet/pods/1fcd5a0f-086a-4958-af2e-e6981691c87b/volumes" Jul 22 19:34:26.539791 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:26.539760 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:34:34.344695 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:34.344660 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd"] Jul 22 19:34:34.345156 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:34.344910 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" containerID="cri-o://75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab" gracePeriod=30 Jul 22 19:34:35.213838 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.213802 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf"] Jul 22 19:34:35.214245 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.214229 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" Jul 22 19:34:35.214319 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.214247 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" Jul 22 19:34:35.214319 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.214275 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" containerName="main" Jul 22 19:34:35.214319 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.214284 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" containerName="main" Jul 22 19:34:35.214475 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.214373 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" Jul 22 19:34:35.214475 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.214388 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3da1b12c-0732-4006-b8b2-6c8c7ac02dc4" containerName="main" Jul 22 19:34:35.219520 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.219494 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.221976 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.221949 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3c150b-openshift-default-dockercfg-gj2x9\"" Jul 22 19:34:35.229003 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.228978 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf"] Jul 22 19:34:35.391978 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.391942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a6e69d8-9583-42c6-8298-419f4290db07-istio-podinfo\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srf7n\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-kube-api-access-srf7n\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-credential-socket\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-certs\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-data\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392206 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-socket\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392303 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-istio-token\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392345 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a6e69d8-9583-42c6-8298-419f4290db07-istiod-ca-cert\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.392433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.392432 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-envoy\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493432 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493332 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-socket\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493432 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-istio-token\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493432 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493399 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a6e69d8-9583-42c6-8298-419f4290db07-istiod-ca-cert\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493432 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493423 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-envoy\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a6e69d8-9583-42c6-8298-419f4290db07-istio-podinfo\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srf7n\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-kube-api-access-srf7n\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-credential-socket\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-certs\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.493841 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493593 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-data\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.494114 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.493848 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-socket\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.494114 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.494004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-data\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.494114 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.494041 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-credential-socket\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.494274 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.494160 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-certs\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.494274 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.494232 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a6e69d8-9583-42c6-8298-419f4290db07-istiod-ca-cert\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.495708 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.495687 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-envoy\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.495958 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.495938 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a6e69d8-9583-42c6-8298-419f4290db07-istio-podinfo\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.502995 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.502967 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-istio-token\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.503232 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.503211 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srf7n\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-kube-api-access-srf7n\") pod \"e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.531729 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.531692 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:35.665998 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.665969 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf"] Jul 22 19:34:35.668597 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:34:35.668569 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6e69d8_9583_42c6_8298_419f4290db07.slice/crio-09eb2fac16605d9d2379738a217b31008dd8f0fee2f13bec104cb41862626f6e WatchSource:0}: Error finding container 09eb2fac16605d9d2379738a217b31008dd8f0fee2f13bec104cb41862626f6e: Status 404 returned error can't find the container with id 09eb2fac16605d9d2379738a217b31008dd8f0fee2f13bec104cb41862626f6e Jul 22 19:34:35.670807 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.670765 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:35.670877 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.670860 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:35.670919 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:35.670905 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:36.603175 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:36.603136 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" event={"ID":"0a6e69d8-9583-42c6-8298-419f4290db07","Type":"ContainerStarted","Data":"f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617"} Jul 22 19:34:36.603568 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:36.603199 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" event={"ID":"0a6e69d8-9583-42c6-8298-419f4290db07","Type":"ContainerStarted","Data":"09eb2fac16605d9d2379738a217b31008dd8f0fee2f13bec104cb41862626f6e"} Jul 22 19:34:36.623094 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:36.623045 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podStartSLOduration=1.6230286459999999 podStartE2EDuration="1.623028646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:34:36.621301723 +0000 UTC m=+2277.347862838" watchObservedRunningTime="2026-07-22 19:34:36.623028646 +0000 UTC m=+2277.349589748" Jul 22 19:34:37.532482 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:37.532440 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:37.537121 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:37.537095 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:37.607739 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:37.607708 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:37.608658 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:37.608641 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:39.605723 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.605697 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:34:39.617691 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.617597 2565 generic.go:356] "Generic (PLEG): container finished" podID="062731b1-13b2-48b2-8043-97a3476c19e7" containerID="75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab" exitCode=0 Jul 22 19:34:39.617691 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.617635 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" event={"ID":"062731b1-13b2-48b2-8043-97a3476c19e7","Type":"ContainerDied","Data":"75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab"} Jul 22 19:34:39.617691 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.617666 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" Jul 22 19:34:39.617923 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.617720 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd" event={"ID":"062731b1-13b2-48b2-8043-97a3476c19e7","Type":"ContainerDied","Data":"9121e937a085d51b1f9d9e67e91cb407bd198d52ceec01bfe99c6fb53f4e61c2"} Jul 22 19:34:39.617923 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.617743 2565 scope.go:117] "RemoveContainer" containerID="75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab" Jul 22 19:34:39.626114 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.626024 2565 scope.go:117] "RemoveContainer" containerID="1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49" Jul 22 19:34:39.629167 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629143 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbwxd\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-kube-api-access-rbwxd\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629289 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629226 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062731b1-13b2-48b2-8043-97a3476c19e7-istio-podinfo\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629289 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629262 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-credential-socket\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629399 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629310 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062731b1-13b2-48b2-8043-97a3476c19e7-istiod-ca-cert\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629399 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629354 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-data\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629399 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629388 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-istio-token\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629544 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629408 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-socket\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629544 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629444 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-certs\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.629544 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.629464 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-envoy\") pod \"062731b1-13b2-48b2-8043-97a3476c19e7\" (UID: \"062731b1-13b2-48b2-8043-97a3476c19e7\") " Jul 22 19:34:39.630708 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.630679 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:39.631328 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.630858 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:39.631328 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.631277 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:39.631328 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.631206 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/062731b1-13b2-48b2-8043-97a3476c19e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:34:39.631534 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.631408 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:39.631889 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.631862 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-kube-api-access-rbwxd" (OuterVolumeSpecName: "kube-api-access-rbwxd") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "kube-api-access-rbwxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:39.632769 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.632743 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/062731b1-13b2-48b2-8043-97a3476c19e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:34:39.633353 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.633332 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:39.633489 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.633470 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "062731b1-13b2-48b2-8043-97a3476c19e7" (UID: "062731b1-13b2-48b2-8043-97a3476c19e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:39.655587 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.655562 2565 scope.go:117] "RemoveContainer" containerID="75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab" Jul 22 19:34:39.655949 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:34:39.655924 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab\": container with ID starting with 75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab not found: ID does not exist" containerID="75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab" Jul 22 19:34:39.656035 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.655963 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab"} err="failed to get container status \"75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab\": rpc error: code = NotFound desc = could not find container \"75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab\": container with ID starting with 75626ab3e1999fcdbcad1d89915c780ba044a09f2bc0ab5722ad8f88fb88c2ab not found: ID does not exist" Jul 22 19:34:39.656035 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.655988 2565 scope.go:117] "RemoveContainer" containerID="1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49" Jul 22 19:34:39.656280 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:34:39.656263 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49\": container with ID starting with 1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49 not found: ID does not exist" containerID="1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49" Jul 22 19:34:39.656323 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.656285 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49"} err="failed to get container status \"1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49\": rpc error: code = NotFound desc = could not find container \"1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49\": container with ID starting with 1e663e1cfa7e8a269b67e03898a238a608161a6d8082df470303dfa7c5a78d49 not found: ID does not exist" Jul 22 19:34:39.731237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731196 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbwxd\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-kube-api-access-rbwxd\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731230 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062731b1-13b2-48b2-8043-97a3476c19e7-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731243 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731528 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731253 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062731b1-13b2-48b2-8043-97a3476c19e7-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731528 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731262 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731528 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731270 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062731b1-13b2-48b2-8043-97a3476c19e7-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731528 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731280 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731528 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731288 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.731528 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.731296 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062731b1-13b2-48b2-8043-97a3476c19e7-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:34:39.933710 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.933675 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd"] Jul 22 19:34:39.937714 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:39.937685 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-dc2828-openshift-default-74874bcc49-wpvbd"] Jul 22 19:34:41.813548 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:41.813513 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" path="/var/lib/kubelet/pods/062731b1-13b2-48b2-8043-97a3476c19e7/volumes" Jul 22 19:34:43.545059 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.545023 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:34:43.746115 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746073 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw"] Jul 22 19:34:43.746512 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746494 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" Jul 22 19:34:43.746601 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746514 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" Jul 22 19:34:43.746601 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746532 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" Jul 22 19:34:43.746601 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746540 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" Jul 22 19:34:43.746780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746654 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" Jul 22 19:34:43.746780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746670 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcd5a0f-086a-4958-af2e-e6981691c87b" containerName="istio-proxy" Jul 22 19:34:43.746780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.746678 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" Jul 22 19:34:43.751121 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.751098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.754321 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.754297 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b3fa31-openshift-default-dockercfg-4jgcm\"" Jul 22 19:34:43.768766 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-credential-socket\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.768898 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768779 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-podinfo\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.768898 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768806 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-envoy\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.768898 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768848 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26efe94e-717d-4a6c-9114-dc73e6a4d595-istiod-ca-cert\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.769007 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7q6\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-kube-api-access-6v7q6\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.769007 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768938 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-token\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.769007 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.768989 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-data\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.769105 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.769025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-socket\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.769105 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.769050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-certs\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.784016 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.783984 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw"] Jul 22 19:34:43.870428 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870323 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-data\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870428 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-socket\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870428 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-certs\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870456 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-credential-socket\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-podinfo\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870511 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-envoy\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26efe94e-717d-4a6c-9114-dc73e6a4d595-istiod-ca-cert\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870677 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7q6\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-kube-api-access-6v7q6\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.870751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-token\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.871012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-data\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.871012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870957 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-certs\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.871012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.870999 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-socket\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.871286 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.871260 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-credential-socket\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.871523 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.871499 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26efe94e-717d-4a6c-9114-dc73e6a4d595-istiod-ca-cert\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.873174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.873156 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-envoy\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.873301 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.873284 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-podinfo\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.885662 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.885634 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-token\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:43.885776 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:43.885659 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7q6\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-kube-api-access-6v7q6\") pod \"e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:44.061508 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.061467 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:44.204075 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.204039 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw"] Jul 22 19:34:44.207015 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:34:44.206980 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26efe94e_717d_4a6c_9114_dc73e6a4d595.slice/crio-2c3d77ae322177948832659467bbd76734d6555b5c918cecef8983f672eca080 WatchSource:0}: Error finding container 2c3d77ae322177948832659467bbd76734d6555b5c918cecef8983f672eca080: Status 404 returned error can't find the container with id 2c3d77ae322177948832659467bbd76734d6555b5c918cecef8983f672eca080 Jul 22 19:34:44.209187 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.209156 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:44.209249 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.209226 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:44.209287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.209254 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:44.648036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.647998 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" event={"ID":"26efe94e-717d-4a6c-9114-dc73e6a4d595","Type":"ContainerStarted","Data":"634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d"} Jul 22 19:34:44.648439 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.648055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" event={"ID":"26efe94e-717d-4a6c-9114-dc73e6a4d595","Type":"ContainerStarted","Data":"2c3d77ae322177948832659467bbd76734d6555b5c918cecef8983f672eca080"} Jul 22 19:34:44.668072 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:44.668009 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podStartSLOduration=1.6679853 podStartE2EDuration="1.6679853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:34:44.666149476 +0000 UTC m=+2285.392710578" watchObservedRunningTime="2026-07-22 19:34:44.6679853 +0000 UTC m=+2285.394546403" Jul 22 19:34:45.064165 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:45.064126 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:45.079297 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:45.079264 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:45.653311 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:45.653280 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:45.654347 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:45.654327 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:34:51.681058 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:51.681025 2565 generic.go:356] "Generic (PLEG): container finished" podID="0a6e69d8-9583-42c6-8298-419f4290db07" containerID="f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617" exitCode=0 Jul 22 19:34:51.681486 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:51.681064 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" event={"ID":"0a6e69d8-9583-42c6-8298-419f4290db07","Type":"ContainerDied","Data":"f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617"} Jul 22 19:34:51.681486 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:51.681469 2565 scope.go:117] "RemoveContainer" containerID="f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617" Jul 22 19:34:51.682506 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:51.682473 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:51.682581 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:51.682563 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:51.682645 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:51.682632 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:34:52.608145 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:52.608101 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:52.689839 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:52.689803 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" event={"ID":"0a6e69d8-9583-42c6-8298-419f4290db07","Type":"ContainerStarted","Data":"88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802"} Jul 22 19:34:53.532917 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:53.532877 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:53.534175 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:53.534149 2565 patch_prober.go:28] interesting pod/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 22 19:34:53.534292 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:53.534201 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 22 19:34:54.532482 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:54.532438 2565 patch_prober.go:28] interesting pod/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 22 19:34:54.532989 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:54.532504 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 22 19:34:54.842271 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:54.842171 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:34:55.532783 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:55.532739 2565 patch_prober.go:28] interesting pod/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 22 19:34:55.533193 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:55.532799 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 22 19:34:56.532720 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:56.532672 2565 patch_prober.go:28] interesting pod/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 22 19:34:56.532914 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:56.532754 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 22 19:34:57.536424 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:57.536393 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:57.536844 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:57.536642 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:34:57.537263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:34:57.537246 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:35:01.694818 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:01.694772 2565 patch_prober.go:28] interesting pod/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:35:01.695313 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:01.694837 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:35:03.746225 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:03.746188 2565 generic.go:356] "Generic (PLEG): container finished" podID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerID="634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d" exitCode=0 Jul 22 19:35:03.746705 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:03.746262 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" event={"ID":"26efe94e-717d-4a6c-9114-dc73e6a4d595","Type":"ContainerDied","Data":"634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d"} Jul 22 19:35:03.746705 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:03.746669 2565 scope.go:117] "RemoveContainer" containerID="634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d" Jul 22 19:35:03.747696 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:03.747672 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:35:03.747766 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:03.747738 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:35:03.747808 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:03.747771 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:35:04.752702 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:04.752666 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" event={"ID":"26efe94e-717d-4a6c-9114-dc73e6a4d595","Type":"ContainerStarted","Data":"4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e"} Jul 22 19:35:05.062258 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:05.062147 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:35:05.062443 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:05.062290 2565 patch_prober.go:28] interesting pod/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:35:05.062443 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:05.062330 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:35:06.062425 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:06.062378 2565 patch_prober.go:28] interesting pod/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:35:06.062973 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:06.062448 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:35:07.062917 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:07.062878 2565 patch_prober.go:28] interesting pod/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:35:07.063945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:07.063524 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:35:08.062180 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:08.062137 2565 patch_prober.go:28] interesting pod/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:35:08.062475 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:08.062446 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:35:09.065649 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:09.065597 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:35:09.066109 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:09.065907 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:35:09.066643 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:09.066626 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:35:17.546394 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:17.546355 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:35:25.555626 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:25.555569 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf"] Jul 22 19:35:25.556220 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:25.555847 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" containerID="cri-o://88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802" gracePeriod=30 Jul 22 19:35:27.538872 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:27.538831 2565 patch_prober.go:28] interesting pod/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:35:27.539295 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:27.538886 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:35:30.804560 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.804525 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:35:30.808974 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.808955 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a6e69d8-9583-42c6-8298-419f4290db07-istiod-ca-cert\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809077 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.808985 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-certs\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809077 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809003 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-istio-token\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809077 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809037 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-socket\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809077 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809077 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a6e69d8-9583-42c6-8298-419f4290db07-istio-podinfo\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809093 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-credential-socket\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809116 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-envoy\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809155 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srf7n\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-kube-api-access-srf7n\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809188 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-data\") pod \"0a6e69d8-9583-42c6-8298-419f4290db07\" (UID: \"0a6e69d8-9583-42c6-8298-419f4290db07\") " Jul 22 19:35:30.809287 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809271 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:30.809530 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809347 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:30.809530 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809394 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:30.809530 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809400 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6e69d8-9583-42c6-8298-419f4290db07-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:35:30.809530 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809502 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.809530 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809522 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.809764 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.809535 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.810218 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.810191 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:30.811338 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.811314 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a6e69d8-9583-42c6-8298-419f4290db07-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:35:30.811538 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.811512 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:35:30.811751 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.811734 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:30.811813 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.811744 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-kube-api-access-srf7n" (OuterVolumeSpecName: "kube-api-access-srf7n") pod "0a6e69d8-9583-42c6-8298-419f4290db07" (UID: "0a6e69d8-9583-42c6-8298-419f4290db07"). InnerVolumeSpecName "kube-api-access-srf7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:35:30.879922 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.879824 2565 generic.go:356] "Generic (PLEG): container finished" podID="0a6e69d8-9583-42c6-8298-419f4290db07" containerID="88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802" exitCode=0 Jul 22 19:35:30.879922 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.879878 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" Jul 22 19:35:30.880174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.879917 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" event={"ID":"0a6e69d8-9583-42c6-8298-419f4290db07","Type":"ContainerDied","Data":"88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802"} Jul 22 19:35:30.880174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.879988 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf" event={"ID":"0a6e69d8-9583-42c6-8298-419f4290db07","Type":"ContainerDied","Data":"09eb2fac16605d9d2379738a217b31008dd8f0fee2f13bec104cb41862626f6e"} Jul 22 19:35:30.880174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.880019 2565 scope.go:117] "RemoveContainer" containerID="88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802" Jul 22 19:35:30.888680 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.888663 2565 scope.go:117] "RemoveContainer" containerID="f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617" Jul 22 19:35:30.897025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.897001 2565 scope.go:117] "RemoveContainer" containerID="88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802" Jul 22 19:35:30.897324 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:35:30.897302 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802\": container with ID starting with 88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802 not found: ID does not exist" containerID="88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802" Jul 22 19:35:30.897405 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.897331 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802"} err="failed to get container status \"88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802\": rpc error: code = NotFound desc = could not find container \"88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802\": container with ID starting with 88a460f87e317a01d6d9e62b131328895d7b0e293e2d32855c1371e6a109e802 not found: ID does not exist" Jul 22 19:35:30.897405 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.897350 2565 scope.go:117] "RemoveContainer" containerID="f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617" Jul 22 19:35:30.897598 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:35:30.897580 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617\": container with ID starting with f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617 not found: ID does not exist" containerID="f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617" Jul 22 19:35:30.897690 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.897621 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617"} err="failed to get container status \"f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617\": rpc error: code = NotFound desc = could not find container \"f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617\": container with ID starting with f44ff7305ef97033cdc7f6855345889907752897bd7f9dfaa41c218ee1f84617 not found: ID does not exist" Jul 22 19:35:30.900722 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.900701 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf"] Jul 22 19:35:30.904671 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.904649 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3c150b-openshift-default-758b4d4755-vt5sf"] Jul 22 19:35:30.910478 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.910457 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a6e69d8-9583-42c6-8298-419f4290db07-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.910580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.910483 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.910580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.910499 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srf7n\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-kube-api-access-srf7n\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.910580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.910511 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a6e69d8-9583-42c6-8298-419f4290db07-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.910580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.910525 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a6e69d8-9583-42c6-8298-419f4290db07-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:30.910580 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:30.910537 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a6e69d8-9583-42c6-8298-419f4290db07-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:31.814342 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:31.814303 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" path="/var/lib/kubelet/pods/0a6e69d8-9583-42c6-8298-419f4290db07/volumes" Jul 22 19:35:46.401655 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:46.401602 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:35:52.174568 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:52.174534 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw"] Jul 22 19:35:52.174989 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:52.174779 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" containerID="cri-o://4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e" gracePeriod=30 Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.050454 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69"] Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051244 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051262 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="062731b1-13b2-48b2-8043-97a3476c19e7" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051303 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051312 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051324 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051334 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051463 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" Jul 22 19:35:53.052679 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.051481 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6e69d8-9583-42c6-8298-419f4290db07" containerName="istio-proxy" Jul 22 19:35:53.055162 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.055129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.057673 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.057636 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-938311-openshift-default-dockercfg-9t2lz\"" Jul 22 19:35:53.061147 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.061123 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69"] Jul 22 19:35:53.110550 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-envoy\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110567 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64kf6\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-kube-api-access-64kf6\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-token\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110691 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-credential-socket\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110754 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110709 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36a1e049-9efa-4d81-8428-b2dd3430bafb-istiod-ca-cert\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110891 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110771 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-certs\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110891 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-socket\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110891 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110881 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-podinfo\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.110988 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.110918 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-data\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212303 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212267 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-envoy\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64kf6\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-kube-api-access-64kf6\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-token\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-credential-socket\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36a1e049-9efa-4d81-8428-b2dd3430bafb-istiod-ca-cert\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212467 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-certs\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-socket\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-podinfo\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.212780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212577 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-data\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.213184 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.212802 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-credential-socket\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.213184 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.213054 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-data\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.213184 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.213093 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-certs\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.213184 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.213165 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-socket\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.213316 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.213207 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36a1e049-9efa-4d81-8428-b2dd3430bafb-istiod-ca-cert\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.214624 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.214582 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-envoy\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.215226 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.215204 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-podinfo\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.222579 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.222555 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-token\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.222729 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.222679 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64kf6\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-kube-api-access-64kf6\") pod \"e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.367833 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.367727 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:53.510456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.510430 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69"] Jul 22 19:35:53.512959 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:35:53.512931 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a1e049_9efa_4d81_8428_b2dd3430bafb.slice/crio-2f6a2fe7a6580859801ee0254ad858b3f8522775a895954288ba1bd35e138a68 WatchSource:0}: Error finding container 2f6a2fe7a6580859801ee0254ad858b3f8522775a895954288ba1bd35e138a68: Status 404 returned error can't find the container with id 2f6a2fe7a6580859801ee0254ad858b3f8522775a895954288ba1bd35e138a68 Jul 22 19:35:53.514999 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.514967 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:35:53.515093 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.515049 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:35:53.515093 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.515080 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:35:53.983158 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.983124 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" event={"ID":"36a1e049-9efa-4d81-8428-b2dd3430bafb","Type":"ContainerStarted","Data":"9356d425067f8d280903473927c0968f4c274ceedc7ea19131aaaebd1775ff1f"} Jul 22 19:35:53.983358 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:53.983190 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" event={"ID":"36a1e049-9efa-4d81-8428-b2dd3430bafb","Type":"ContainerStarted","Data":"2f6a2fe7a6580859801ee0254ad858b3f8522775a895954288ba1bd35e138a68"} Jul 22 19:35:54.003252 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.003198 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podStartSLOduration=1.003179141 podStartE2EDuration="1.003179141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:35:54.002432216 +0000 UTC m=+2354.728993319" watchObservedRunningTime="2026-07-22 19:35:54.003179141 +0000 UTC m=+2354.729740221" Jul 22 19:35:54.068742 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.068704 2565 patch_prober.go:28] interesting pod/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:35:54.068920 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.068761 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:35:54.368581 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.368482 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:54.373711 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.373678 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:54.988193 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.988162 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:54.989237 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:54.989217 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:35:57.426886 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.426859 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:35:57.549174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549075 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26efe94e-717d-4a6c-9114-dc73e6a4d595-istiod-ca-cert\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549174 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549137 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-data\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549176 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v7q6\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-kube-api-access-6v7q6\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549206 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-podinfo\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549271 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-token\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549299 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-socket\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549334 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-certs\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549367 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-credential-socket\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549483 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549393 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-envoy\") pod \"26efe94e-717d-4a6c-9114-dc73e6a4d595\" (UID: \"26efe94e-717d-4a6c-9114-dc73e6a4d595\") " Jul 22 19:35:57.549880 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549535 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26efe94e-717d-4a6c-9114-dc73e6a4d595-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:35:57.549880 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549831 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26efe94e-717d-4a6c-9114-dc73e6a4d595-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.549880 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549830 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:57.550052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.549895 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:57.550052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.550023 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:57.550052 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.550038 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-data" (OuterVolumeSpecName: "istio-data") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:57.551471 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.551445 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:35:57.551627 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.551592 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:35:57.551913 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.551895 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-token" (OuterVolumeSpecName: "istio-token") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:35:57.551956 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.551939 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-kube-api-access-6v7q6" (OuterVolumeSpecName: "kube-api-access-6v7q6") pod "26efe94e-717d-4a6c-9114-dc73e6a4d595" (UID: "26efe94e-717d-4a6c-9114-dc73e6a4d595"). InnerVolumeSpecName "kube-api-access-6v7q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:35:57.650592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650544 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v7q6\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-kube-api-access-6v7q6\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650582 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650595 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650632 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650917 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650644 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650917 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650656 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650917 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650666 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:57.650917 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:57.650677 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26efe94e-717d-4a6c-9114-dc73e6a4d595-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:35:58.004281 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.004246 2565 generic.go:356] "Generic (PLEG): container finished" podID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerID="4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e" exitCode=0 Jul 22 19:35:58.004456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.004300 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" event={"ID":"26efe94e-717d-4a6c-9114-dc73e6a4d595","Type":"ContainerDied","Data":"4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e"} Jul 22 19:35:58.004456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.004305 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" Jul 22 19:35:58.004456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.004343 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw" event={"ID":"26efe94e-717d-4a6c-9114-dc73e6a4d595","Type":"ContainerDied","Data":"2c3d77ae322177948832659467bbd76734d6555b5c918cecef8983f672eca080"} Jul 22 19:35:58.004456 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.004367 2565 scope.go:117] "RemoveContainer" containerID="4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e" Jul 22 19:35:58.012589 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.012572 2565 scope.go:117] "RemoveContainer" containerID="634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d" Jul 22 19:35:58.020508 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.020490 2565 scope.go:117] "RemoveContainer" containerID="4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e" Jul 22 19:35:58.020830 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:35:58.020802 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e\": container with ID starting with 4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e not found: ID does not exist" containerID="4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e" Jul 22 19:35:58.020976 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.020841 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e"} err="failed to get container status \"4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e\": rpc error: code = NotFound desc = could not find container \"4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e\": container with ID starting with 4ead9ccdda3c84a5eeab4ad2510198130219639eae7f16a38e1682dc8b5e674e not found: ID does not exist" Jul 22 19:35:58.021043 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.020982 2565 scope.go:117] "RemoveContainer" containerID="634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d" Jul 22 19:35:58.021443 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:35:58.021397 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d\": container with ID starting with 634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d not found: ID does not exist" containerID="634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d" Jul 22 19:35:58.021546 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.021442 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d"} err="failed to get container status \"634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d\": rpc error: code = NotFound desc = could not find container \"634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d\": container with ID starting with 634bfe819701baa8f680e9e93351f5ba2dc0fa8f001ab5f7a212cf7fecee5d3d not found: ID does not exist" Jul 22 19:35:58.022219 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.022199 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw"] Jul 22 19:35:58.023545 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:58.023526 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b3fa31-openshift-default-fb9486477-9d5bw"] Jul 22 19:35:59.814192 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:35:59.814159 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" path="/var/lib/kubelet/pods/26efe94e-717d-4a6c-9114-dc73e6a4d595/volumes" Jul 22 19:36:00.967470 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.967432 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt"] Jul 22 19:36:00.967997 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.967975 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" Jul 22 19:36:00.968048 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.968003 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" Jul 22 19:36:00.968125 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.968114 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" Jul 22 19:36:00.968167 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.968129 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" Jul 22 19:36:00.971150 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.971127 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:00.973903 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.973880 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4addf8-openshift-default-dockercfg-ltdh6\"" Jul 22 19:36:00.982420 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:00.982395 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt"] Jul 22 19:36:01.046317 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.046276 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:36:01.082141 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082093 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-socket\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-envoy\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082264 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-certs\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082298 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-data\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082323 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/469f280c-7ceb-40c6-b826-0d2282eecd0d-istiod-ca-cert\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082351 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcqsr\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-kube-api-access-tcqsr\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082654 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082385 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-token\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082654 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082414 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-credential-socket\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.082654 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.082458 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-podinfo\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183491 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-socket\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183491 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183495 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-envoy\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-certs\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-data\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/469f280c-7ceb-40c6-b826-0d2282eecd0d-istiod-ca-cert\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcqsr\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-kube-api-access-tcqsr\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-token\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-credential-socket\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.183778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183754 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-podinfo\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.184127 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.183962 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-data\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.184127 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.184045 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-certs\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.184563 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.184506 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-credential-socket\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.184563 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.184515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-socket\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.184741 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.184641 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/469f280c-7ceb-40c6-b826-0d2282eecd0d-istiod-ca-cert\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.186708 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.186671 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-envoy\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.186843 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.186831 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-podinfo\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.191901 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.191872 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-token\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.192163 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.192141 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcqsr\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-kube-api-access-tcqsr\") pod \"e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.282743 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.282645 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:01.432581 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.432557 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt"] Jul 22 19:36:01.434769 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:36:01.434743 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469f280c_7ceb_40c6_b826_0d2282eecd0d.slice/crio-cffaa126ceffb52ec7025629ce6716782f3cb4bf0bb46653c9c962f09411d816 WatchSource:0}: Error finding container cffaa126ceffb52ec7025629ce6716782f3cb4bf0bb46653c9c962f09411d816: Status 404 returned error can't find the container with id cffaa126ceffb52ec7025629ce6716782f3cb4bf0bb46653c9c962f09411d816 Jul 22 19:36:01.437360 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.437328 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:01.437414 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.437401 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:01.437449 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:01.437430 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:02.023953 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:02.023919 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" event={"ID":"469f280c-7ceb-40c6-b826-0d2282eecd0d","Type":"ContainerStarted","Data":"549dbc8b50ba11f4b7aafeceb0c3a2c7823d03ca25d4a8da04529849c0810273"} Jul 22 19:36:02.024358 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:02.023975 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" event={"ID":"469f280c-7ceb-40c6-b826-0d2282eecd0d","Type":"ContainerStarted","Data":"cffaa126ceffb52ec7025629ce6716782f3cb4bf0bb46653c9c962f09411d816"} Jul 22 19:36:02.044251 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:02.044195 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podStartSLOduration=2.044177156 podStartE2EDuration="2.044177156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:36:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:36:02.042214539 +0000 UTC m=+2362.768775654" watchObservedRunningTime="2026-07-22 19:36:02.044177156 +0000 UTC m=+2362.770738258" Jul 22 19:36:02.284801 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:02.283274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:02.290401 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:02.290371 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:03.029409 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:03.029372 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:03.030292 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:03.030274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:09.989385 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:09.989349 2565 patch_prober.go:28] interesting pod/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 22 19:36:09.989844 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:09.989415 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 22 19:36:10.062445 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.062414 2565 generic.go:356] "Generic (PLEG): container finished" podID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerID="9356d425067f8d280903473927c0968f4c274ceedc7ea19131aaaebd1775ff1f" exitCode=0 Jul 22 19:36:10.062656 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.062464 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" event={"ID":"36a1e049-9efa-4d81-8428-b2dd3430bafb","Type":"ContainerDied","Data":"9356d425067f8d280903473927c0968f4c274ceedc7ea19131aaaebd1775ff1f"} Jul 22 19:36:10.062876 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.062859 2565 scope.go:117] "RemoveContainer" containerID="9356d425067f8d280903473927c0968f4c274ceedc7ea19131aaaebd1775ff1f" Jul 22 19:36:10.063942 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.063913 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:10.064001 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.063980 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:10.064036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.064011 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:10.142882 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:10.142841 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:36:11.072136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:11.072084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" event={"ID":"36a1e049-9efa-4d81-8428-b2dd3430bafb","Type":"ContainerStarted","Data":"2da671139f8a502e265735f85aae548cfdab82ef937728ebd0c54ace13faa629"} Jul 22 19:36:11.369188 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:11.369084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:36:11.369374 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:11.369229 2565 patch_prober.go:28] interesting pod/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 22 19:36:11.369374 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:11.369275 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 22 19:36:12.369006 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:12.368958 2565 patch_prober.go:28] interesting pod/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 22 19:36:12.369557 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:12.369031 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 22 19:36:13.368313 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:13.368267 2565 patch_prober.go:28] interesting pod/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 22 19:36:13.368532 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:13.368337 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 22 19:36:14.368847 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:14.368798 2565 patch_prober.go:28] interesting pod/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 22 19:36:14.369383 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:14.368872 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 22 19:36:15.378950 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:15.378912 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:36:15.381047 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:15.381021 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:36:15.385071 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:15.385044 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:36:18.490767 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:18.490210 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:55060->10.134.0.45:15021: read: connection reset by peer" start-of-body= Jul 22 19:36:18.490767 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:18.490284 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:55060->10.134.0.45:15021: read: connection reset by peer" Jul 22 19:36:19.116949 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:19.116915 2565 generic.go:356] "Generic (PLEG): container finished" podID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerID="549dbc8b50ba11f4b7aafeceb0c3a2c7823d03ca25d4a8da04529849c0810273" exitCode=0 Jul 22 19:36:19.116949 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:19.116954 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" event={"ID":"469f280c-7ceb-40c6-b826-0d2282eecd0d","Type":"ContainerDied","Data":"549dbc8b50ba11f4b7aafeceb0c3a2c7823d03ca25d4a8da04529849c0810273"} Jul 22 19:36:19.117359 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:19.117345 2565 scope.go:117] "RemoveContainer" containerID="549dbc8b50ba11f4b7aafeceb0c3a2c7823d03ca25d4a8da04529849c0810273" Jul 22 19:36:19.118324 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:19.118291 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:19.118395 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:19.118358 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:19.118395 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:19.118390 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:36:20.123411 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:20.123376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" event={"ID":"469f280c-7ceb-40c6-b826-0d2282eecd0d","Type":"ContainerStarted","Data":"5993c4dd532a37ce240f4bde2862b877c4e20f2fe6fa71ba5de1b36b92acad53"} Jul 22 19:36:20.283466 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:20.283429 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:20.283725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:20.283703 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:36:20.283778 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:20.283764 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:36:21.283844 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:21.283792 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:36:21.284348 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:21.283880 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:36:22.284011 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:22.283904 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:36:22.284656 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:22.284594 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:36:23.286634 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:23.286089 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:36:23.286634 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:23.286189 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:36:24.283597 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:24.283549 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:36:24.283838 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:24.283641 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:36:25.287209 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:25.287169 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:25.287738 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:25.287575 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:25.288313 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:25.288296 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:36:40.443166 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:36:40.443128 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:37:12.341440 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:37:12.341399 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:37:36.040948 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:37:36.040908 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:37:54.948072 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:37:54.948038 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:38:14.243259 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:38:14.243221 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:38:30.651250 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:38:30.651216 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:38:49.744916 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:38:49.744837 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:39:08.938758 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:08.938719 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:39:23.944276 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:23.944243 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:39:30.751622 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.751563 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq"] Jul 22 19:39:30.752062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.751926 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" Jul 22 19:39:30.752062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.751938 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26efe94e-717d-4a6c-9114-dc73e6a4d595" containerName="istio-proxy" Jul 22 19:39:30.753910 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.753892 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.756433 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.756407 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-938311\"/\"openshift-service-ca.crt\"" Jul 22 19:39:30.757308 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.757289 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-938311\"/\"kube-root-ca.crt\"" Jul 22 19:39:30.757308 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.757301 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-938311\"/\"default-dockercfg-ch9n6\"" Jul 22 19:39:30.757472 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.757302 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-938311\"/\"sub-test-model-938311-kserve-self-signed-certs\"" Jul 22 19:39:30.763095 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.763069 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq"] Jul 22 19:39:30.863692 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.863653 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tmp-dir\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.863890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.863711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jd6q\" (UniqueName: \"kubernetes.io/projected/044361d6-1f7d-4088-b68b-d4fa2fb5756f-kube-api-access-2jd6q\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.863890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.863842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-dshm\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.863976 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.863899 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-home\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.863976 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.863919 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tls-certs\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.863976 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.863952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-model-cache\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.965572 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.965531 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tmp-dir\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.965812 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.965625 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jd6q\" (UniqueName: \"kubernetes.io/projected/044361d6-1f7d-4088-b68b-d4fa2fb5756f-kube-api-access-2jd6q\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.965812 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.965670 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-dshm\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.965812 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.965733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-home\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.965812 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.965758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tls-certs\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.965812 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.965786 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-model-cache\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.966161 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.966135 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tmp-dir\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.966250 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.966223 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-model-cache\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.966714 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.966692 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-home\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.969449 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.969258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-dshm\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.969449 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.969369 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tls-certs\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:30.977647 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:30.977572 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jd6q\" (UniqueName: \"kubernetes.io/projected/044361d6-1f7d-4088-b68b-d4fa2fb5756f-kube-api-access-2jd6q\") pod \"sub-test-model-938311-kserve-598ddf9989-szmtq\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:31.063453 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:31.063355 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:31.202014 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:31.201951 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq"] Jul 22 19:39:31.204323 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:39:31.204290 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod044361d6_1f7d_4088_b68b_d4fa2fb5756f.slice/crio-754d513adae55dfe8a4648ed3d34712f009b0bb52556c6e51698608479640573 WatchSource:0}: Error finding container 754d513adae55dfe8a4648ed3d34712f009b0bb52556c6e51698608479640573: Status 404 returned error can't find the container with id 754d513adae55dfe8a4648ed3d34712f009b0bb52556c6e51698608479640573 Jul 22 19:39:31.980771 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:31.980733 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" event={"ID":"044361d6-1f7d-4088-b68b-d4fa2fb5756f","Type":"ContainerStarted","Data":"f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4"} Jul 22 19:39:31.980771 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:31.980777 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" event={"ID":"044361d6-1f7d-4088-b68b-d4fa2fb5756f","Type":"ContainerStarted","Data":"754d513adae55dfe8a4648ed3d34712f009b0bb52556c6e51698608479640573"} Jul 22 19:39:31.999366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:31.999310 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" podStartSLOduration=1.999293707 podStartE2EDuration="1.999293707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:39:31.997678605 +0000 UTC m=+2572.724239707" watchObservedRunningTime="2026-07-22 19:39:31.999293707 +0000 UTC m=+2572.725854809" Jul 22 19:39:40.639227 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:40.639177 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:39:41.063730 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:41.063686 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:41.063936 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:41.063747 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:41.076791 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:41.076762 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:42.037582 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:42.037545 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:39:57.137451 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:39:57.137413 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:40:27.649169 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:40:27.649125 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:40:45.049655 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:40:45.049601 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq"] Jul 22 19:40:50.546363 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:40:50.546329 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:41:00.070088 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.070042 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" podUID="044361d6-1f7d-4088-b68b-d4fa2fb5756f" containerName="main" containerID="cri-o://f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4" gracePeriod=45 Jul 22 19:41:00.316805 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.316780 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:41:00.378138 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.378048 2565 generic.go:356] "Generic (PLEG): container finished" podID="044361d6-1f7d-4088-b68b-d4fa2fb5756f" containerID="f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4" exitCode=0 Jul 22 19:41:00.378138 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.378110 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" Jul 22 19:41:00.378138 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.378131 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" event={"ID":"044361d6-1f7d-4088-b68b-d4fa2fb5756f","Type":"ContainerDied","Data":"f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4"} Jul 22 19:41:00.378354 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.378176 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq" event={"ID":"044361d6-1f7d-4088-b68b-d4fa2fb5756f","Type":"ContainerDied","Data":"754d513adae55dfe8a4648ed3d34712f009b0bb52556c6e51698608479640573"} Jul 22 19:41:00.378354 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.378201 2565 scope.go:117] "RemoveContainer" containerID="f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4" Jul 22 19:41:00.389127 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.387889 2565 scope.go:117] "RemoveContainer" containerID="f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4" Jul 22 19:41:00.389264 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:41:00.389212 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4\": container with ID starting with f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4 not found: ID does not exist" containerID="f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4" Jul 22 19:41:00.389264 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.389241 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4"} err="failed to get container status \"f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4\": rpc error: code = NotFound desc = could not find container \"f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4\": container with ID starting with f50ac4f4688e830f8dae80df076db99a649e7f6125560d299a560daa334e62a4 not found: ID does not exist" Jul 22 19:41:00.440119 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440088 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tls-certs\") pod \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " Jul 22 19:41:00.440322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440134 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-home\") pod \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " Jul 22 19:41:00.440322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440154 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-model-cache\") pod \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " Jul 22 19:41:00.440322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440182 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tmp-dir\") pod \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " Jul 22 19:41:00.440322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440211 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-dshm\") pod \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " Jul 22 19:41:00.440322 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440263 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jd6q\" (UniqueName: \"kubernetes.io/projected/044361d6-1f7d-4088-b68b-d4fa2fb5756f-kube-api-access-2jd6q\") pod \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\" (UID: \"044361d6-1f7d-4088-b68b-d4fa2fb5756f\") " Jul 22 19:41:00.440592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440422 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-model-cache" (OuterVolumeSpecName: "model-cache") pod "044361d6-1f7d-4088-b68b-d4fa2fb5756f" (UID: "044361d6-1f7d-4088-b68b-d4fa2fb5756f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:41:00.440592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440439 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-home" (OuterVolumeSpecName: "home") pod "044361d6-1f7d-4088-b68b-d4fa2fb5756f" (UID: "044361d6-1f7d-4088-b68b-d4fa2fb5756f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:41:00.440592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440509 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "044361d6-1f7d-4088-b68b-d4fa2fb5756f" (UID: "044361d6-1f7d-4088-b68b-d4fa2fb5756f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:41:00.440592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440564 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-home\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:41:00.440592 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.440583 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-model-cache\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:41:00.442417 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.442373 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-dshm" (OuterVolumeSpecName: "dshm") pod "044361d6-1f7d-4088-b68b-d4fa2fb5756f" (UID: "044361d6-1f7d-4088-b68b-d4fa2fb5756f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:41:00.442417 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.442384 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "044361d6-1f7d-4088-b68b-d4fa2fb5756f" (UID: "044361d6-1f7d-4088-b68b-d4fa2fb5756f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:41:00.442554 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.442482 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044361d6-1f7d-4088-b68b-d4fa2fb5756f-kube-api-access-2jd6q" (OuterVolumeSpecName: "kube-api-access-2jd6q") pod "044361d6-1f7d-4088-b68b-d4fa2fb5756f" (UID: "044361d6-1f7d-4088-b68b-d4fa2fb5756f"). InnerVolumeSpecName "kube-api-access-2jd6q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:41:00.541674 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.541636 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tls-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:41:00.541674 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.541684 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-tmp-dir\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:41:00.541890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.541693 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/044361d6-1f7d-4088-b68b-d4fa2fb5756f-dshm\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:41:00.541890 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.541702 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jd6q\" (UniqueName: \"kubernetes.io/projected/044361d6-1f7d-4088-b68b-d4fa2fb5756f-kube-api-access-2jd6q\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:41:00.700257 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.700227 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq"] Jul 22 19:41:00.704055 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.704032 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-938311/sub-test-model-938311-kserve-598ddf9989-szmtq"] Jul 22 19:41:00.937582 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:00.937497 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:41:01.813858 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:01.813824 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="044361d6-1f7d-4088-b68b-d4fa2fb5756f" path="/var/lib/kubelet/pods/044361d6-1f7d-4088-b68b-d4fa2fb5756f/volumes" Jul 22 19:41:21.947376 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.947336 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk"] Jul 22 19:41:21.949809 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.947743 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044361d6-1f7d-4088-b68b-d4fa2fb5756f" containerName="main" Jul 22 19:41:21.949809 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.947760 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="044361d6-1f7d-4088-b68b-d4fa2fb5756f" containerName="main" Jul 22 19:41:21.949809 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.947832 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="044361d6-1f7d-4088-b68b-d4fa2fb5756f" containerName="main" Jul 22 19:41:21.950895 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.950880 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:21.953176 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.953153 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4addf8\"/\"kube-root-ca.crt\"" Jul 22 19:41:21.954135 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.954116 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4addf8\"/\"rate-test-model-4addf8-kserve-self-signed-certs\"" Jul 22 19:41:21.954190 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.954116 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4addf8\"/\"default-dockercfg-xvs67\"" Jul 22 19:41:21.954190 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.954155 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4addf8\"/\"openshift-service-ca.crt\"" Jul 22 19:41:21.963725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:21.963699 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk"] Jul 22 19:41:22.024953 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.024918 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-home\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.024953 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.024956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-dshm\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.025205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.024983 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvf9m\" (UniqueName: \"kubernetes.io/projected/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-kube-api-access-bvf9m\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.025205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.025035 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-model-cache\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.025205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.025118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tls-certs\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.025205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.025145 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tmp-dir\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.125780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.125744 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tls-certs\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.125780 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.125782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tmp-dir\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.125854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-home\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.125881 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-dshm\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.125927 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvf9m\" (UniqueName: \"kubernetes.io/projected/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-kube-api-access-bvf9m\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126036 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.125952 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-model-cache\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126298 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.126269 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-home\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126434 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.126315 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tmp-dir\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.126515 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.126484 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-model-cache\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.128767 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.128736 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-dshm\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.128896 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.128847 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tls-certs\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.133686 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.133667 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvf9m\" (UniqueName: \"kubernetes.io/projected/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-kube-api-access-bvf9m\") pod \"rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.259504 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.259419 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:22.394444 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.394416 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk"] Jul 22 19:41:22.397248 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:41:22.397220 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod142b40b4_5d94_4e80_8f50_ba1d9e77b9ee.slice/crio-8b721bc85a68dde568973e60ea47968e3513f4e2b7c18bc961dbd68d1dbbfc13 WatchSource:0}: Error finding container 8b721bc85a68dde568973e60ea47968e3513f4e2b7c18bc961dbd68d1dbbfc13: Status 404 returned error can't find the container with id 8b721bc85a68dde568973e60ea47968e3513f4e2b7c18bc961dbd68d1dbbfc13 Jul 22 19:41:22.485506 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.485468 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" event={"ID":"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee","Type":"ContainerStarted","Data":"06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a"} Jul 22 19:41:22.485725 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.485518 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" event={"ID":"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee","Type":"ContainerStarted","Data":"8b721bc85a68dde568973e60ea47968e3513f4e2b7c18bc961dbd68d1dbbfc13"} Jul 22 19:41:22.502445 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:22.502399 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" podStartSLOduration=1.5023819619999998 podStartE2EDuration="1.502381962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:41:22.500769917 +0000 UTC m=+2683.227331039" watchObservedRunningTime="2026-07-22 19:41:22.502381962 +0000 UTC m=+2683.228943097" Jul 22 19:41:25.635235 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:25.635199 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:41:32.260504 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:32.260410 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:32.260944 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:32.260499 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:32.273373 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:32.273347 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:32.543028 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:32.542947 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:41:43.341247 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:43.341205 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:41:59.248093 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:41:59.248055 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:42:07.315025 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:07.314978 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk"] Jul 22 19:42:14.540210 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:14.540172 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:42:22.336753 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.336679 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" podUID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" containerName="main" containerID="cri-o://06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a" gracePeriod=45 Jul 22 19:42:22.583286 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.583259 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:42:22.665378 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665347 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-dshm\") pod \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " Jul 22 19:42:22.665378 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665382 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tls-certs\") pod \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " Jul 22 19:42:22.665635 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665399 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tmp-dir\") pod \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " Jul 22 19:42:22.665635 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665523 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-home\") pod \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " Jul 22 19:42:22.665635 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665569 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvf9m\" (UniqueName: \"kubernetes.io/projected/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-kube-api-access-bvf9m\") pod \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " Jul 22 19:42:22.665635 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665617 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-model-cache\") pod \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\" (UID: \"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee\") " Jul 22 19:42:22.665853 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665681 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" (UID: "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:22.665853 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665795 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-home" (OuterVolumeSpecName: "home") pod "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" (UID: "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:22.665954 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665910 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" (UID: "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:22.665954 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665934 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-home\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:22.666042 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.665980 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tmp-dir\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:22.667876 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.667853 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-kube-api-access-bvf9m" (OuterVolumeSpecName: "kube-api-access-bvf9m") pod "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" (UID: "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee"). InnerVolumeSpecName "kube-api-access-bvf9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:22.667987 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.667946 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-dshm" (OuterVolumeSpecName: "dshm") pod "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" (UID: "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:22.668055 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.667996 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" (UID: "142b40b4-5d94-4e80-8f50-ba1d9e77b9ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:42:22.754019 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.753987 2565 generic.go:356] "Generic (PLEG): container finished" podID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" containerID="06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a" exitCode=0 Jul 22 19:42:22.754198 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.754030 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" Jul 22 19:42:22.754198 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.754071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" event={"ID":"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee","Type":"ContainerDied","Data":"06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a"} Jul 22 19:42:22.754198 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.754115 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" event={"ID":"142b40b4-5d94-4e80-8f50-ba1d9e77b9ee","Type":"ContainerDied","Data":"8b721bc85a68dde568973e60ea47968e3513f4e2b7c18bc961dbd68d1dbbfc13"} Jul 22 19:42:22.754198 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.754134 2565 scope.go:117] "RemoveContainer" containerID="06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a" Jul 22 19:42:22.763266 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.763249 2565 scope.go:117] "RemoveContainer" containerID="06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a" Jul 22 19:42:22.763533 ip-10-0-137-176 kubenswrapper[2565]: E0722 19:42:22.763515 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a\": container with ID starting with 06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a not found: ID does not exist" containerID="06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a" Jul 22 19:42:22.763597 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.763543 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a"} err="failed to get container status \"06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a\": rpc error: code = NotFound desc = could not find container \"06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a\": container with ID starting with 06f6738b37647716cd0872f820bbfd3670acab6be2ddf111fe7c3c27a3c2717a not found: ID does not exist" Jul 22 19:42:22.766473 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.766458 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-dshm\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:22.766537 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.766475 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-tls-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:22.766537 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.766485 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvf9m\" (UniqueName: \"kubernetes.io/projected/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-kube-api-access-bvf9m\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:22.766537 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.766495 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee-model-cache\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:22.774126 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.774105 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk"] Jul 22 19:42:22.777786 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:22.777762 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk"] Jul 22 19:42:23.813294 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:23.813250 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" path="/var/lib/kubelet/pods/142b40b4-5d94-4e80-8f50-ba1d9e77b9ee/volumes" Jul 22 19:42:27.532056 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:27.532015 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-4addf8/rate-test-model-4addf8-kserve-6cf68b4ccf-qxznk" podUID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded" Jul 22 19:42:39.642944 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:39.642907 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:42:47.688104 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:47.688067 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69"] Jul 22 19:42:47.688727 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:47.688289 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" containerID="cri-o://2da671139f8a502e265735f85aae548cfdab82ef937728ebd0c54ace13faa629" gracePeriod=30 Jul 22 19:42:52.890574 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:52.890539 2565 generic.go:356] "Generic (PLEG): container finished" podID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerID="2da671139f8a502e265735f85aae548cfdab82ef937728ebd0c54ace13faa629" exitCode=0 Jul 22 19:42:52.891256 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:52.890626 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" event={"ID":"36a1e049-9efa-4d81-8428-b2dd3430bafb","Type":"ContainerDied","Data":"2da671139f8a502e265735f85aae548cfdab82ef937728ebd0c54ace13faa629"} Jul 22 19:42:52.891256 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:52.890693 2565 scope.go:117] "RemoveContainer" containerID="9356d425067f8d280903473927c0968f4c274ceedc7ea19131aaaebd1775ff1f" Jul 22 19:42:52.945058 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:52.945034 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:42:53.025910 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.025809 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-token\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.025910 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.025878 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-certs\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.025919 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-envoy\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.025956 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36a1e049-9efa-4d81-8428-b2dd3430bafb-istiod-ca-cert\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026012 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-credential-socket\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026062 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64kf6\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-kube-api-access-64kf6\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026088 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-podinfo\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026117 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-socket\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026458 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026143 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-data\") pod \"36a1e049-9efa-4d81-8428-b2dd3430bafb\" (UID: \"36a1e049-9efa-4d81-8428-b2dd3430bafb\") " Jul 22 19:42:53.026458 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026220 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:53.026458 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026335 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:53.026640 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026486 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.026640 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026473 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:53.026640 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026506 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.026640 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.026504 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a1e049-9efa-4d81-8428-b2dd3430bafb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:42:53.027170 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.027146 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-data" (OuterVolumeSpecName: "istio-data") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:53.028329 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.028298 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:42:53.028435 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.028367 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-kube-api-access-64kf6" (OuterVolumeSpecName: "kube-api-access-64kf6") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "kube-api-access-64kf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:53.028503 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.028486 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:53.028594 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.028576 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-token" (OuterVolumeSpecName: "istio-token") pod "36a1e049-9efa-4d81-8428-b2dd3430bafb" (UID: "36a1e049-9efa-4d81-8428-b2dd3430bafb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:53.127372 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127333 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.127372 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127366 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36a1e049-9efa-4d81-8428-b2dd3430bafb-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.127372 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127377 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64kf6\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-kube-api-access-64kf6\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.127659 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127387 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.127659 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127396 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.127659 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127404 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.127659 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.127412 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36a1e049-9efa-4d81-8428-b2dd3430bafb-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:42:53.896838 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.896804 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" event={"ID":"36a1e049-9efa-4d81-8428-b2dd3430bafb","Type":"ContainerDied","Data":"2f6a2fe7a6580859801ee0254ad858b3f8522775a895954288ba1bd35e138a68"} Jul 22 19:42:53.897248 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.896858 2565 scope.go:117] "RemoveContainer" containerID="2da671139f8a502e265735f85aae548cfdab82ef937728ebd0c54ace13faa629" Jul 22 19:42:53.897248 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.896819 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69" Jul 22 19:42:53.916541 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.916513 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69"] Jul 22 19:42:53.922417 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:53.922386 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-938311-openshift-default-59bb4d8567-mqp69"] Jul 22 19:42:55.813624 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:42:55.813579 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" path="/var/lib/kubelet/pods/36a1e049-9efa-4d81-8428-b2dd3430bafb/volumes" Jul 22 19:43:01.338622 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:01.338525 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xmgc9"] Jul 22 19:43:09.784266 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:09.784219 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt"] Jul 22 19:43:09.784710 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:09.784517 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" containerID="cri-o://5993c4dd532a37ce240f4bde2862b877c4e20f2fe6fa71ba5de1b36b92acad53" gracePeriod=30 Jul 22 19:43:10.289526 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:10.289491 2565 patch_prober.go:28] interesting pod/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:43:10.289719 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:10.289570 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:43:10.461938 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:10.461909 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-696b684d48-t22v7_5370e11e-53f9-44ba-8317-b1a39a739901/authorino/0.log" Jul 22 19:43:14.998001 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:14.997963 2565 generic.go:356] "Generic (PLEG): container finished" podID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerID="5993c4dd532a37ce240f4bde2862b877c4e20f2fe6fa71ba5de1b36b92acad53" exitCode=0 Jul 22 19:43:14.998473 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:14.998031 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" event={"ID":"469f280c-7ceb-40c6-b826-0d2282eecd0d","Type":"ContainerDied","Data":"5993c4dd532a37ce240f4bde2862b877c4e20f2fe6fa71ba5de1b36b92acad53"} Jul 22 19:43:14.998473 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:14.998126 2565 scope.go:117] "RemoveContainer" containerID="549dbc8b50ba11f4b7aafeceb0c3a2c7823d03ca25d4a8da04529849c0810273" Jul 22 19:43:15.036263 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.036234 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:43:15.114962 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.114870 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-data\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.114962 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.114921 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-envoy\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.114962 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.114942 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-credential-socket\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.114981 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-certs\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115008 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcqsr\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-kube-api-access-tcqsr\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115042 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-podinfo\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115069 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/469f280c-7ceb-40c6-b826-0d2282eecd0d-istiod-ca-cert\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115121 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-socket\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115243 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115143 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-token\") pod \"469f280c-7ceb-40c6-b826-0d2282eecd0d\" (UID: \"469f280c-7ceb-40c6-b826-0d2282eecd0d\") " Jul 22 19:43:15.115559 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115299 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:43:15.115559 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115350 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:43:15.115559 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115461 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-credential-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.115559 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115478 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-certs\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.115804 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115594 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/469f280c-7ceb-40c6-b826-0d2282eecd0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:43:15.115804 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115778 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:43:15.115954 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.115920 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:43:15.117191 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.117159 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:43:15.117496 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.117472 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:43:15.117710 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.117694 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:43:15.117870 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.117848 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-kube-api-access-tcqsr" (OuterVolumeSpecName: "kube-api-access-tcqsr") pod "469f280c-7ceb-40c6-b826-0d2282eecd0d" (UID: "469f280c-7ceb-40c6-b826-0d2282eecd0d"). InnerVolumeSpecName "kube-api-access-tcqsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:43:15.216393 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216353 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-podinfo\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.216547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216406 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/469f280c-7ceb-40c6-b826-0d2282eecd0d-istiod-ca-cert\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.216547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216416 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-workload-socket\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.216547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216424 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-token\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.216547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216432 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-data\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.216547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216439 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/469f280c-7ceb-40c6-b826-0d2282eecd0d-istio-envoy\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:15.216547 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:15.216448 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcqsr\" (UniqueName: \"kubernetes.io/projected/469f280c-7ceb-40c6-b826-0d2282eecd0d-kube-api-access-tcqsr\") on node \"ip-10-0-137-176.ec2.internal\" DevicePath \"\"" Jul 22 19:43:16.004214 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.004177 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" event={"ID":"469f280c-7ceb-40c6-b826-0d2282eecd0d","Type":"ContainerDied","Data":"cffaa126ceffb52ec7025629ce6716782f3cb4bf0bb46653c9c962f09411d816"} Jul 22 19:43:16.004214 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.004210 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt" Jul 22 19:43:16.004214 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.004242 2565 scope.go:117] "RemoveContainer" containerID="5993c4dd532a37ce240f4bde2862b877c4e20f2fe6fa71ba5de1b36b92acad53" Jul 22 19:43:16.020987 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.020956 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt"] Jul 22 19:43:16.026222 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.026195 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4addf8-openshift-default-5c466484f5-8x6kt"] Jul 22 19:43:16.463411 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.463383 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb586fcb5-qtv66_37dd099c-c1ea-4c86-9bf7-e927cffbb93e/copy-manifests/0.log" Jul 22 19:43:16.469892 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:16.469866 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb586fcb5-qtv66_37dd099c-c1ea-4c86-9bf7-e927cffbb93e/manager/0.log" Jul 22 19:43:17.813681 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:17.813648 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" path="/var/lib/kubelet/pods/469f280c-7ceb-40c6-b826-0d2282eecd0d/volumes" Jul 22 19:43:18.523111 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:18.523080 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-696b684d48-t22v7_5370e11e-53f9-44ba-8317-b1a39a739901/authorino/0.log" Jul 22 19:43:18.642713 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:18.642688 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mpjnj_70685a16-0c64-46ed-8bac-cd573e834559/manager/0.log" Jul 22 19:43:18.753315 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:18.753292 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bzl2f_f30038ef-4037-4c39-b698-8e9c05b0cea6/manager/0.log" Jul 22 19:43:18.976551 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:18.976521 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kzvht_647acce4-20c3-484e-a712-8a44ac928bb8/registry-server/0.log" Jul 22 19:43:19.212167 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:19.212124 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xmgc9_05187f41-0418-4725-a92c-d30adba3bd03/limitador/0.log" Jul 22 19:43:19.669416 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:19.669388 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nq7wx_a01c936f-1571-44fd-aad9-cede33668e20/istio-proxy/0.log" Jul 22 19:43:20.128564 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:20.128473 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/1.log" Jul 22 19:43:21.154367 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:21.154335 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb5whd_9571595d-86b0-4375-bd26-13531a186cca/main/0.log" Jul 22 19:43:26.570791 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.570754 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mw4wd/must-gather-d7swk"] Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571198 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571226 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571252 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571260 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571270 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" containerName="main" Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571279 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" containerName="main" Jul 22 19:43:26.571290 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571291 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571299 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571315 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571324 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571413 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571425 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571437 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="142b40b4-5d94-4e80-8f50-ba1d9e77b9ee" containerName="main" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571592 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="469f280c-7ceb-40c6-b826-0d2282eecd0d" containerName="istio-proxy" Jul 22 19:43:26.571699 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.571621 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a1e049-9efa-4d81-8428-b2dd3430bafb" containerName="istio-proxy" Jul 22 19:43:26.574577 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.574556 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.577014 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.576971 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mw4wd\"/\"default-dockercfg-lwfnl\"" Jul 22 19:43:26.577205 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.577192 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mw4wd\"/\"openshift-service-ca.crt\"" Jul 22 19:43:26.577855 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.577835 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mw4wd\"/\"kube-root-ca.crt\"" Jul 22 19:43:26.584168 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.584145 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mw4wd/must-gather-d7swk"] Jul 22 19:43:26.613735 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.613697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6nft\" (UniqueName: \"kubernetes.io/projected/efd44e10-82a5-4e57-9e38-11a9efd08267-kube-api-access-g6nft\") pod \"must-gather-d7swk\" (UID: \"efd44e10-82a5-4e57-9e38-11a9efd08267\") " pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.613923 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.613773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/efd44e10-82a5-4e57-9e38-11a9efd08267-must-gather-output\") pod \"must-gather-d7swk\" (UID: \"efd44e10-82a5-4e57-9e38-11a9efd08267\") " pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.714749 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.714707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6nft\" (UniqueName: \"kubernetes.io/projected/efd44e10-82a5-4e57-9e38-11a9efd08267-kube-api-access-g6nft\") pod \"must-gather-d7swk\" (UID: \"efd44e10-82a5-4e57-9e38-11a9efd08267\") " pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.714929 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.714775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/efd44e10-82a5-4e57-9e38-11a9efd08267-must-gather-output\") pod \"must-gather-d7swk\" (UID: \"efd44e10-82a5-4e57-9e38-11a9efd08267\") " pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.715056 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.715042 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/efd44e10-82a5-4e57-9e38-11a9efd08267-must-gather-output\") pod \"must-gather-d7swk\" (UID: \"efd44e10-82a5-4e57-9e38-11a9efd08267\") " pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.722930 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.722912 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6nft\" (UniqueName: \"kubernetes.io/projected/efd44e10-82a5-4e57-9e38-11a9efd08267-kube-api-access-g6nft\") pod \"must-gather-d7swk\" (UID: \"efd44e10-82a5-4e57-9e38-11a9efd08267\") " pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:26.884855 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:26.884756 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mw4wd/must-gather-d7swk" Jul 22 19:43:27.016694 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:27.016586 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mw4wd/must-gather-d7swk"] Jul 22 19:43:27.019159 ip-10-0-137-176 kubenswrapper[2565]: W0722 19:43:27.019129 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefd44e10_82a5_4e57_9e38_11a9efd08267.slice/crio-4c0fb0949b98db0c10303afd4e0b3acbba69ff99613941c935ff1f75cce5dbdb WatchSource:0}: Error finding container 4c0fb0949b98db0c10303afd4e0b3acbba69ff99613941c935ff1f75cce5dbdb: Status 404 returned error can't find the container with id 4c0fb0949b98db0c10303afd4e0b3acbba69ff99613941c935ff1f75cce5dbdb Jul 22 19:43:27.021376 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:27.021362 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:43:27.054136 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:27.054101 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mw4wd/must-gather-d7swk" event={"ID":"efd44e10-82a5-4e57-9e38-11a9efd08267","Type":"ContainerStarted","Data":"4c0fb0949b98db0c10303afd4e0b3acbba69ff99613941c935ff1f75cce5dbdb"} Jul 22 19:43:28.062013 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:28.061918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mw4wd/must-gather-d7swk" event={"ID":"efd44e10-82a5-4e57-9e38-11a9efd08267","Type":"ContainerStarted","Data":"9d1f44269389a11d431a494d4a238ff4a05eee9c4651e8833f72b17367b66b1e"} Jul 22 19:43:28.062013 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:28.061976 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mw4wd/must-gather-d7swk" event={"ID":"efd44e10-82a5-4e57-9e38-11a9efd08267","Type":"ContainerStarted","Data":"1b63661581257a412b4a4794a1e36fd325529750a2b65f39ac6f37e9feab75df"} Jul 22 19:43:28.080046 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:28.079822 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mw4wd/must-gather-d7swk" podStartSLOduration=1.411724786 podStartE2EDuration="2.079798681s" totalImagesPullingTime="668.073895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:43:28.078155806 +0000 UTC m=+2808.804716911" watchObservedRunningTime="2026-07-22 19:43:28.079798681 +0000 UTC m=+2808.806359783" Jul 22 19:43:29.221955 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:29.221926 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5qpnv_e7f38ebc-93c7-4813-9b3f-3038502b046b/global-pull-secret-syncer/0.log" Jul 22 19:43:29.346012 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:29.345980 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n96vv_2e083d91-f5fa-42ce-b755-872ff84c3af8/konnectivity-agent/0.log" Jul 22 19:43:29.415881 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:29.415847 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-176.ec2.internal_10d7c501939fb84cb4667bd33ca0ad06/haproxy/0.log" Jul 22 19:43:34.692281 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:34.692241 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-696b684d48-t22v7_5370e11e-53f9-44ba-8317-b1a39a739901/authorino/0.log" Jul 22 19:43:34.722572 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:34.722546 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mpjnj_70685a16-0c64-46ed-8bac-cd573e834559/manager/0.log" Jul 22 19:43:34.757487 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:34.757450 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bzl2f_f30038ef-4037-4c39-b698-8e9c05b0cea6/manager/0.log" Jul 22 19:43:34.829153 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:34.829105 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kzvht_647acce4-20c3-484e-a712-8a44ac928bb8/registry-server/0.log" Jul 22 19:43:35.074536 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:35.074384 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xmgc9_05187f41-0418-4725-a92c-d30adba3bd03/limitador/0.log" Jul 22 19:43:37.127463 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:37.127427 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-798d7cd4d9-xfzv7_a5466ae1-d2b9-4842-9145-b50c3223b816/metrics-server/0.log" Jul 22 19:43:37.263667 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:37.263542 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-czvpw_dc485f2a-051f-4af7-87c7-36296e7430b1/node-exporter/0.log" Jul 22 19:43:37.285076 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:37.284979 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-czvpw_dc485f2a-051f-4af7-87c7-36296e7430b1/kube-rbac-proxy/0.log" Jul 22 19:43:37.305096 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:37.305072 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-czvpw_dc485f2a-051f-4af7-87c7-36296e7430b1/init-textfile/0.log" Jul 22 19:43:39.234980 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.234948 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm"] Jul 22 19:43:39.239828 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.239801 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.245177 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.245152 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm"] Jul 22 19:43:39.343824 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.343790 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dngkm\" (UniqueName: \"kubernetes.io/projected/9acc114b-644c-429c-b327-b9413ec33a50-kube-api-access-dngkm\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.344040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.343843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-sys\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.344040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.343914 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-proc\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.344040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.343952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-podres\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.344040 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.343977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-lib-modules\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.445537 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.445487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-sys\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.445763 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.445557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-proc\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.445763 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.445597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-podres\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.445763 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.445657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-lib-modules\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.445763 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.445749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dngkm\" (UniqueName: \"kubernetes.io/projected/9acc114b-644c-429c-b327-b9413ec33a50-kube-api-access-dngkm\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.446216 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.446185 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-sys\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.446311 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.446266 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-proc\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.446389 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.446376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-podres\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.446487 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.446465 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9acc114b-644c-429c-b327-b9413ec33a50-lib-modules\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.454028 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.453998 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dngkm\" (UniqueName: \"kubernetes.io/projected/9acc114b-644c-429c-b327-b9413ec33a50-kube-api-access-dngkm\") pod \"perf-node-gather-daemonset-khrdm\" (UID: \"9acc114b-644c-429c-b327-b9413ec33a50\") " pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.553113 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.552822 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:39.737947 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:39.736821 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm"] Jul 22 19:43:40.151727 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:40.151642 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" event={"ID":"9acc114b-644c-429c-b327-b9413ec33a50","Type":"ContainerStarted","Data":"b97183e035c103f38593b8f243c401cc07b1e7e77163897cc395a97603b80f9b"} Jul 22 19:43:40.151727 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:40.151718 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" event={"ID":"9acc114b-644c-429c-b327-b9413ec33a50","Type":"ContainerStarted","Data":"e5baa3f67b4fe50e5b39aebcf5932630f5ba28d4da9431ac397923c9e1773635"} Jul 22 19:43:40.152112 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:40.152088 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:40.168315 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:40.168253 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" podStartSLOduration=1.168228784 podStartE2EDuration="1.168228784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 19:43:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:43:40.166687828 +0000 UTC m=+2820.893248960" watchObservedRunningTime="2026-07-22 19:43:40.168228784 +0000 UTC m=+2820.894789887" Jul 22 19:43:41.849379 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:41.849351 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dfkvn_a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb/dns/0.log" Jul 22 19:43:41.868762 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:41.868721 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dfkvn_a8a3c7ae-6b73-47b3-ac4f-2365fb5cdedb/kube-rbac-proxy/0.log" Jul 22 19:43:41.996477 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:41.996438 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c9786_c8da430a-be95-4881-9b8c-25b984a290e7/dns-node-resolver/0.log" Jul 22 19:43:42.598945 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:42.598905 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mkx5n_8ae751a3-8d23-4553-ac23-3e752a6958b7/node-ca/0.log" Jul 22 19:43:43.518154 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:43.518121 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nq7wx_a01c936f-1571-44fd-aad9-cede33668e20/istio-proxy/0.log" Jul 22 19:43:43.707738 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:43.707705 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/0.log" Jul 22 19:43:43.743851 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:43.743824 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/1.log" Jul 22 19:43:44.482472 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:44.482444 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x589_7578bff6-22d9-4bd1-981f-0fccb3b3bf8a/serve-healthcheck-canary/0.log" Jul 22 19:43:45.115366 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:45.115318 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-flr6f_6859a523-ead0-4896-bd3a-826f9998e504/kube-rbac-proxy/0.log" Jul 22 19:43:45.134049 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:45.134020 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-flr6f_6859a523-ead0-4896-bd3a-826f9998e504/exporter/0.log" Jul 22 19:43:45.154265 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:45.154240 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-flr6f_6859a523-ead0-4896-bd3a-826f9998e504/extractor/0.log" Jul 22 19:43:46.167466 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:46.167438 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mw4wd/perf-node-gather-daemonset-khrdm" Jul 22 19:43:47.644004 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:47.643970 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb586fcb5-qtv66_37dd099c-c1ea-4c86-9bf7-e927cffbb93e/manager/0.log" Jul 22 19:43:47.664321 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:47.664298 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cb586fcb5-qtv66_37dd099c-c1ea-4c86-9bf7-e927cffbb93e/copy-manifests/0.log" Jul 22 19:43:49.225829 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:49.225761 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dd84c44d8-p2jnh_3319b9d1-894f-4f36-b1e9-cde27f62b721/manager/0.log" Jul 22 19:43:50.361096 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:50.361061 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nq7wx_a01c936f-1571-44fd-aad9-cede33668e20/istio-proxy/0.log" Jul 22 19:43:50.525221 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:50.525146 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/0.log" Jul 22 19:43:50.564928 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:50.564896 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/1.log" Jul 22 19:43:51.778351 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:51.778318 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nq7wx_a01c936f-1571-44fd-aad9-cede33668e20/istio-proxy/0.log" Jul 22 19:43:51.953316 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:51.953273 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/0.log" Jul 22 19:43:51.991988 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:51.991942 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9cscd_ac6da53a-b447-4006-9def-52442f5e25c9/istio-proxy/1.log" Jul 22 19:43:54.155805 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:54.155759 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-t4bjv_b45ec387-cde4-4b3e-9ad6-742c19bef34c/migrator/0.log" Jul 22 19:43:54.175705 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:54.175669 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-t4bjv_b45ec387-cde4-4b3e-9ad6-742c19bef34c/graceful-termination/0.log" Jul 22 19:43:55.432274 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.432243 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/kube-multus-additional-cni-plugins/0.log" Jul 22 19:43:55.451568 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.451544 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/egress-router-binary-copy/0.log" Jul 22 19:43:55.473178 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.473147 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/cni-plugins/0.log" Jul 22 19:43:55.492562 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.492537 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/bond-cni-plugin/0.log" Jul 22 19:43:55.510459 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.510423 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/routeoverride-cni/0.log" Jul 22 19:43:55.528062 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.528037 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/whereabouts-cni-bincopy/0.log" Jul 22 19:43:55.546531 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.546499 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bt5vk_7922174c-9acd-4037-afd8-d26f3745b099/whereabouts-cni/0.log" Jul 22 19:43:55.866493 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.866420 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n52sf_3f4d5757-a5e8-4a05-bde9-4fe1897cfe74/kube-multus/0.log" Jul 22 19:43:55.992489 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:55.992458 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vnnnq_6e4e1837-2baa-4c5d-927b-70682f840b7a/network-metrics-daemon/0.log" Jul 22 19:43:56.009710 ip-10-0-137-176 kubenswrapper[2565]: I0722 19:43:56.009672 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vnnnq_6e4e1837-2baa-4c5d-927b-70682f840b7a/kube-rbac-proxy/0.log"