Jul 14 09:26:27.887042 ip-10-0-139-42 systemd[1]: Starting Kubernetes Kubelet... Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.336076 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343897 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343912 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343917 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343920 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343923 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343926 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343928 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:28.417381 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343931 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343934 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343937 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343940 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343943 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343945 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343948 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343951 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343962 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343965 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343967 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343970 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343972 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343975 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343977 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343980 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343983 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343985 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343988 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343992 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:28.421993 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343996 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.343999 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344002 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344005 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344008 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344010 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344013 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344016 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344018 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344021 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344024 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344028 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344030 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344033 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344036 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344038 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344041 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344044 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344046 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344049 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344051 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:28.423072 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344054 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:28.423179 ip-10-0-139-42 systemd[1]: Started Kubernetes Kubelet. Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344056 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344059 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344062 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344064 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344067 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344069 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344072 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344075 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344078 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344080 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344082 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344085 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344087 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344090 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344093 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344095 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344104 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344107 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:28.425484 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344111 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344115 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344118 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344122 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344126 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344130 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344133 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344135 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344138 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344141 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344144 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344146 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344149 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344152 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344154 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344156 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344160 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344162 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344165 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344167 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:28.426245 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344169 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344172 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344174 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344177 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344179 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344182 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344184 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344187 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344189 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344193 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344196 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344199 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344933 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344941 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344944 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344947 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344950 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344959 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344962 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:28.427401 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344965 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344968 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344971 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344975 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344979 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344981 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344984 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344986 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344989 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344991 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344994 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344996 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.344999 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345001 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345004 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345006 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345009 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345011 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345014 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345016 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:28.428368 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345019 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345022 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345025 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345027 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345030 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345032 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345035 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345037 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345040 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345042 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345045 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345055 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345057 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345061 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345063 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345067 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345071 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345074 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345076 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:28.429822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345079 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345082 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345085 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345087 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345090 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345093 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345096 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345098 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345101 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345103 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345105 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345108 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345110 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345113 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345116 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345118 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345121 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345124 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345126 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:28.430698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345129 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345131 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345134 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345136 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345139 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345141 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345144 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345147 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345152 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345155 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345157 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345160 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345162 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345165 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345167 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345170 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345172 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345175 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345178 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:28.431784 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345181 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345183 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345186 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345188 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345191 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345194 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345196 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345199 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345201 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345204 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345207 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345210 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345212 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345214 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345217 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345617 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345622 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345625 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345627 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345630 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:28.432514 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345634 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345638 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345641 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345644 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345647 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345649 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345652 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345655 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345657 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345660 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345663 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345666 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345668 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345671 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345673 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345676 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345679 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345681 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345684 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345686 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:28.433354 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345689 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345692 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345695 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345697 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345700 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345703 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345706 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345708 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345729 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345732 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345735 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345738 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345741 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345744 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345746 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345749 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345752 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345755 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345757 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:28.434081 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345760 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345762 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345766 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345769 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345772 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345776 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345778 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345781 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345784 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345786 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345789 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345792 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345794 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345797 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345800 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345803 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345805 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345808 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345811 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:28.435025 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345813 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345818 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345821 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345824 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345827 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345830 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345834 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345837 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345839 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345842 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345844 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345847 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345849 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345852 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345854 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345857 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345859 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345862 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345865 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345867 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:28.435647 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345870 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345872 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345875 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345878 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345880 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345882 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345888 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345890 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345893 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345895 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345898 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345901 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345903 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345905 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345910 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.345913 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.345994 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346001 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346008 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346012 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346017 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 09:26:28.496599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346020 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346024 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346029 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346032 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346035 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346038 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346041 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346045 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346048 2579 flags.go:64] FLAG: --cgroup-root="" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346051 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346054 2579 flags.go:64] FLAG: --client-ca-file="" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346057 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346060 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346064 2579 flags.go:64] FLAG: --cluster-domain="" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346067 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346070 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346073 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346078 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346082 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346085 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346088 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346092 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346095 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346098 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 09:26:28.497678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346101 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346105 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346108 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346113 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346116 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346119 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346121 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346124 2579 flags.go:64] FLAG: --enable-server="true" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346128 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346133 2579 flags.go:64] FLAG: --event-burst="100" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346136 2579 flags.go:64] FLAG: --event-qps="50" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346140 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346143 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346146 2579 flags.go:64] FLAG: --eviction-hard="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346150 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346153 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346156 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346159 2579 flags.go:64] FLAG: --eviction-soft="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346162 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346165 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346168 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346171 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346173 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346176 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346179 2579 flags.go:64] FLAG: --feature-gates="" Jul 14 09:26:28.498874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346185 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346188 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346191 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346194 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346198 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346201 2579 flags.go:64] FLAG: --help="false" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346204 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346207 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346211 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346214 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346217 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346220 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346224 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346227 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346230 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346233 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346236 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346239 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346242 2579 flags.go:64] FLAG: --kube-reserved="" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346245 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346248 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346251 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346254 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346257 2579 flags.go:64] FLAG: --lock-file="" Jul 14 09:26:28.502200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346260 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346262 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346265 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346271 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346274 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346277 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346280 2579 flags.go:64] FLAG: --logging-format="text" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346283 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346287 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346290 2579 flags.go:64] FLAG: --manifest-url="" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346293 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346303 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346306 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346310 2579 flags.go:64] FLAG: --max-pods="110" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346312 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346316 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346320 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346323 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346326 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346329 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346332 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346339 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346342 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346345 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 09:26:28.503463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346348 2579 flags.go:64] FLAG: --pod-cidr="" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346352 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346358 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346361 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346364 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346367 2579 flags.go:64] FLAG: --port="10250" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346370 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346373 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f9e91d3cdd2d246d" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346376 2579 flags.go:64] FLAG: --qos-reserved="" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346379 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346382 2579 flags.go:64] FLAG: --register-node="true" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346385 2579 flags.go:64] FLAG: --register-with-taints="" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346388 2579 flags.go:64] FLAG: --registry-burst="10" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346391 2579 flags.go:64] FLAG: --registry-qps="5" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346394 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346397 2579 flags.go:64] FLAG: --reserved-memory="" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346401 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346411 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346425 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346428 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346431 2579 flags.go:64] FLAG: --runonce="false" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346434 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346437 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346440 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346445 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 09:26:28.504546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346448 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346451 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346454 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346457 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346460 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346463 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346466 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346469 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346472 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346475 2579 flags.go:64] FLAG: --system-cgroups="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346478 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346488 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346491 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346493 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346498 2579 flags.go:64] FLAG: --tls-min-version="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346500 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346503 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346506 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346509 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346512 2579 flags.go:64] FLAG: --v="2" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346521 2579 flags.go:64] FLAG: --version="false" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346525 2579 flags.go:64] FLAG: --vmodule="" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346529 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.346533 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346652 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:28.506690 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346656 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346659 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346662 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346665 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346668 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346671 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346674 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346677 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346679 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346682 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346686 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346690 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346692 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346695 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346697 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346700 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346703 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346705 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346708 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:28.507976 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346727 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346731 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346733 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346736 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346738 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346741 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346744 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346746 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346749 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346751 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346754 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346756 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346761 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346763 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346766 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346769 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346772 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346774 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346777 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346781 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:28.510132 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346783 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346786 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346788 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346791 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346794 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346796 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346799 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346801 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346804 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346807 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346809 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346812 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346814 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346817 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346820 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346822 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346825 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346827 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346830 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346832 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:28.511010 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346835 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346839 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346842 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346845 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346850 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346852 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346855 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346858 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346862 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346864 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346867 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346871 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346874 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346876 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346879 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346882 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346885 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346887 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346890 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346892 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:28.641021 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346895 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346897 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346901 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346904 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346907 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346911 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346914 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346917 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346920 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346922 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346925 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346928 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346930 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346933 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346936 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346939 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346942 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346945 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:28.661938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.346947 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.347708 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.355213 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.355230 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.355254 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.355260 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355311 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355316 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355320 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355323 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355327 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355330 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355333 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355336 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355339 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:28.663218 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355342 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355345 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355348 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355350 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355353 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355355 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355358 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355360 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355363 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355366 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355368 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355370 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355373 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355375 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355378 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355381 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355383 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355386 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355388 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355391 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:28.663686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355394 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355396 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355399 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355404 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355407 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355409 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355412 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355415 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355418 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355420 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355423 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355425 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355428 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355430 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355433 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355435 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355439 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355443 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355446 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:28.664286 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355448 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355451 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355453 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355457 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355461 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355463 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355467 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355470 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355472 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355475 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355478 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355481 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355484 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355486 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355489 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355491 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355495 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355498 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:28.664938 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355500 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355503 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355506 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355508 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355511 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355513 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355516 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355519 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355521 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355524 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355526 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355528 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355531 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355533 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355536 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355538 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355541 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355544 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355546 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355549 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:28.665453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355551 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355554 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355557 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355561 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355564 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355568 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355571 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355573 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355575 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355578 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355582 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355585 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355587 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.355592 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355701 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:28.666090 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355705 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355708 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355728 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355733 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355737 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355742 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355745 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355750 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355753 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355755 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355758 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355761 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355765 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355770 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355772 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355775 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355778 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355781 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355784 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:28.666544 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355787 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355790 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355793 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355795 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355798 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355801 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355803 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355806 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355810 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355813 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355816 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355818 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355821 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355823 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355826 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355828 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355831 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355833 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355836 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355838 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:28.768771 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355842 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355844 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355847 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355849 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355852 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355854 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355857 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355859 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355862 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355865 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355867 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355870 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355872 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355875 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355877 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355880 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355882 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355886 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355890 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:28.769667 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355892 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355896 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355899 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355901 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355904 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355906 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355909 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355911 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355914 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355917 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355919 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355922 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355924 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355928 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355931 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355933 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355935 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355938 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355940 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355943 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:28.770262 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355946 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355948 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355951 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355953 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355956 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355958 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355961 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355963 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355967 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355971 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355973 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355976 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355978 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355983 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355985 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355988 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355991 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355993 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355996 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:28.770782 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:28.355998 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.356003 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.356124 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.360760 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.362531 2579 server.go:1019] "Starting client certificate rotation" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.362631 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.362665 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.365386 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.371908 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.387164 2579 log.go:25] "Validated CRI v1 runtime API" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.389955 2579 log.go:25] "Validated CRI v1 image API" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.390790 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 09:26:28.771275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.393168 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 09:26:28.771593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.394480 2579 fs.go:135] Filesystem UUIDs: map[427120de-cdf8-41b6-add9-23f7e9a72862:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e81bf1a8-9771-4c4d-aa84-28075bc06448:/dev/nvme0n1p3] Jul 14 09:26:28.771593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.394495 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.400581 2579 manager.go:216] Machine: {Timestamp:2026-07-14 09:26:28.399104253 +0000 UTC m=+0.380987355 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101162 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20cb7af13754e63b70b25f646ab8eb SystemUUID:ec20cb7a-f137-54e6-3b70-b25f646ab8eb BootID:048c7432-beec-457e-8047-94bf788fdc4d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c5:ee:2b:4a:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c5:ee:2b:4a:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:60:82:ce:df:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.400683 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.400776 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.404360 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.404394 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-42.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.405294 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.405305 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.405313 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.405328 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.407200 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.411802 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.411978 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412007 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412026 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412041 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412053 2579 kubelet.go:398] "Adding apiserver pod source" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412063 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 09:26:28.771664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412494 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.412515 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.414922 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417405 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417456 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417708 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417750 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417762 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417771 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417784 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.417792 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.418348 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.418356 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.418363 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.418369 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.418377 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.418385 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.419226 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.419236 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.422448 2579 server.go:1268] "Started kubelet" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.422826 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.427339 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.427986 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.429407 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.431066 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.431988 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.432008 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-42.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.432128 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.434588 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.436343 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.436424 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.436451 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.437296 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.437373 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.437589 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.437776 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.437785 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438255 2579 factory.go:55] Registering systemd factory Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438272 2579 factory.go:223] Registration of the systemd container factory successfully Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438517 2579 factory.go:153] Registering CRI-O factory Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438532 2579 factory.go:223] Registration of the crio container factory successfully Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438580 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438602 2579 factory.go:103] Registering Raw factory Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.438617 2579 manager.go:1196] Started watching for new ooms in manager Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.439304 2579 manager.go:349] Starting recovery of all containers Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.445659 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhzf4" Jul 14 09:26:28.772179 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.446046 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.446134 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.446236 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-42.ec2.internal.18c21dd197b06501 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-42.ec2.internal,UID:ip-10-0-139-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-42.ec2.internal,},FirstTimestamp:2026-07-14 09:26:28.422427905 +0000 UTC m=+0.404310988,LastTimestamp:2026-07-14 09:26:28.422427905 +0000 UTC m=+0.404310988,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-42.ec2.internal,}" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.449518 2579 manager.go:354] Recovery completed Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.453583 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhzf4" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.455086 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.456967 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.456987 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.456996 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.457583 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.457593 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.457606 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.457911 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.458636 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.458657 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.458669 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.459180 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-42.ec2.internal.18c21dd199bf9839 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-42.ec2.internal,UID:ip-10-0-139-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-42.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-42.ec2.internal,},FirstTimestamp:2026-07-14 09:26:28.456978489 +0000 UTC m=+0.438861572,LastTimestamp:2026-07-14 09:26:28.456978489 +0000 UTC m=+0.438861572,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-42.ec2.internal,}" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.468778 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-42.ec2.internal.18c21dd199bfca68 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-42.ec2.internal,UID:ip-10-0-139-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-42.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-42.ec2.internal,},FirstTimestamp:2026-07-14 09:26:28.456991336 +0000 UTC m=+0.438874418,LastTimestamp:2026-07-14 09:26:28.456991336 +0000 UTC m=+0.438874418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-42.ec2.internal,}" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.483338 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-42.ec2.internal.18c21dd199bfebc8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-42.ec2.internal,UID:ip-10-0-139-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-42.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-42.ec2.internal,},FirstTimestamp:2026-07-14 09:26:28.45699988 +0000 UTC m=+0.438882962,LastTimestamp:2026-07-14 09:26:28.45699988 +0000 UTC m=+0.438882962,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-42.ec2.internal,}" Jul 14 09:26:28.773374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.487377 2579 policy_none.go:49] "None policy: Start" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.487473 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.487486 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.489574 2579 policy_none.go:47] "Start" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.536936 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.540910 2579 manager.go:337] "Starting Device Plugin manager" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.540942 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.540951 2579 server.go:87] "Starting device plugin registration server" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.541217 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.541228 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.541527 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.541580 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.541585 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.542258 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.542299 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.584606 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.585875 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.585915 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.585940 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.585979 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.588300 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.641945 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.642950 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.642982 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.642996 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.643034 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.653548 2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-42.ec2.internal\" not found" node="ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.655338 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.655360 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-42.ec2.internal\": node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.684745 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.686757 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal"] Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.686840 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.688654 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.688687 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.688699 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.691259 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.691341 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.691367 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.692074 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.692098 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.692107 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.692072 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.692162 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.692175 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.773905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.694427 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.694451 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.695221 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.695250 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.695265 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.713351 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-42.ec2.internal\" not found" node="ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.717978 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-42.ec2.internal\" not found" node="ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.739070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06089b923710f2d2feae9e9599b422d3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal\" (UID: \"06089b923710f2d2feae9e9599b422d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.739100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06089b923710f2d2feae9e9599b422d3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal\" (UID: \"06089b923710f2d2feae9e9599b422d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.775006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:28.739121 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1edd94a745f603ae25b0ce87a74b4849-config\") pod \"kube-apiserver-proxy-ip-10-0-139-42.ec2.internal\" (UID: \"1edd94a745f603ae25b0ce87a74b4849\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" Jul 14 09:26:28.785881 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:28.785846 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:29.884545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:29.884262 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06089b923710f2d2feae9e9599b422d3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal\" (UID: \"06089b923710f2d2feae9e9599b422d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:29.884545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:29.884213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06089b923710f2d2feae9e9599b422d3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal\" (UID: \"06089b923710f2d2feae9e9599b422d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:29.884545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:29.884331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06089b923710f2d2feae9e9599b422d3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal\" (UID: \"06089b923710f2d2feae9e9599b422d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:29.884545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:29.884355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1edd94a745f603ae25b0ce87a74b4849-config\") pod \"kube-apiserver-proxy-ip-10-0-139-42.ec2.internal\" (UID: \"1edd94a745f603ae25b0ce87a74b4849\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" Jul 14 09:26:29.884545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:29.884382 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1edd94a745f603ae25b0ce87a74b4849-config\") pod \"kube-apiserver-proxy-ip-10-0-139-42.ec2.internal\" (UID: \"1edd94a745f603ae25b0ce87a74b4849\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" Jul 14 09:26:29.884545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:29.884412 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06089b923710f2d2feae9e9599b422d3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal\" (UID: \"06089b923710f2d2feae9e9599b422d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:29.930547 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:29.930504 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:30.031124 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.031096 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:30.060306 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.060280 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:30.066725 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.066705 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" Jul 14 09:26:30.131527 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.131479 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:30.231976 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.231939 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:30.332478 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.332392 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:30.406055 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.406019 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 09:26:30.406209 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.406188 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 09:26:30.433311 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.433285 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-42.ec2.internal\" not found" Jul 14 09:26:30.451552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.451528 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:30.456536 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.456521 2579 apiserver.go:52] "Watching apiserver" Jul 14 09:26:30.467119 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.467094 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 09:26:30.469158 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.468610 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8q6mb","kube-system/konnectivity-agent-brzt6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l","openshift-dns/node-resolver-zxfnl","openshift-image-registry/node-ca-cglm5","openshift-multus/multus-additional-cni-plugins-2g8sm","openshift-ovn-kubernetes/ovnkube-node-4bjh8","openshift-cluster-node-tuning-operator/tuned-4kwn8","openshift-multus/multus-2mxt2","openshift-multus/network-metrics-daemon-fpqzv","openshift-network-diagnostics/network-check-target-8tlqr","openshift-network-operator/iptables-alerter-tjqrx"] Jul 14 09:26:30.471483 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.471462 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.473639 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.473623 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.474320 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.474300 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.474320 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.474310 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 09:26:30.474433 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.474316 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.474433 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.474308 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sc9bh\"" Jul 14 09:26:30.475637 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.475619 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.475861 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.475845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.475926 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.475878 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.476325 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.476310 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-45wtk\"" Jul 14 09:26:30.478042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.478027 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.478160 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.478141 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-68859\"" Jul 14 09:26:30.478778 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.478764 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 09:26:30.478821 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.478809 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 09:26:30.478915 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.478902 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.480197 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.480186 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:30.480261 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.480246 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:30.480454 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.480441 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.480492 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.480482 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 09:26:30.480632 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.480618 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.481063 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.481049 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.481127 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.481115 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gzgp7\"" Jul 14 09:26:30.481238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.481226 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 09:26:30.481353 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.481342 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" Jul 14 09:26:30.482336 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.482322 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:30.482392 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.482377 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:30.484462 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.484450 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.486607 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.486589 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.486680 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.486661 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:30.487638 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487622 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-system-cni-dir\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.487680 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487646 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysctl-conf\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487680 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487668 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-lib-modules\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-tmp\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-device-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.487739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487712 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysctl-d\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-systemd\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-run\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-socket-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.487887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487868 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-modprobe-d\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.487887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-registration-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8g2l\" (UniqueName: \"kubernetes.io/projected/36e51c26-bfe7-4fa9-ae78-3703c1e59027-kube-api-access-c8g2l\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59d7060b-7f01-4a2f-9e55-503176c06e81-hosts-file\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-sys\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.487996 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-tuned\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srgrb\" (UniqueName: \"kubernetes.io/projected/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-kube-api-access-srgrb\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488056 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86rnb\"" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcd6f\" (UniqueName: \"kubernetes.io/projected/1bcf77f5-947b-4432-b408-425e3e0612a6-kube-api-access-vcd6f\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:30.488121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488104 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-sys-fs\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d7060b-7f01-4a2f-9e55-503176c06e81-tmp-dir\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-cnibin\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-os-release\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488239 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-etc-selinux\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488258 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488294 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysconfig\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488323 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-kubernetes\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488351 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-var-lib-kubelet\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488384 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m2xg\" (UniqueName: \"kubernetes.io/projected/59d7060b-7f01-4a2f-9e55-503176c06e81-kube-api-access-2m2xg\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.488431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488434 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b6ms\" (UniqueName: \"kubernetes.io/projected/61a507c5-2f51-4ba3-9866-6e7397b160f4-kube-api-access-8b6ms\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488860 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488458 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-host\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.488860 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:30.488860 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488495 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.488991 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.488976 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.489360 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.489346 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.491140 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.491127 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.493520 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.493504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.495694 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.495678 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.499664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.499641 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 09:26:30.499736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.499698 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 09:26:30.500772 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.500742 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 09:26:30.500823 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.500774 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.500941 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.500929 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.502064 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.502044 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 09:26:30.502064 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.502057 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 09:26:30.502295 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.502079 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 09:26:30.511525 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.511509 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 09:26:30.511593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.511509 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 09:26:30.512558 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.512540 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 09:26:30.512558 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.512552 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7h9hb\"" Jul 14 09:26:30.512694 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.512616 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9wzq8\"" Jul 14 09:26:30.512694 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.512628 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wmlhg\"" Jul 14 09:26:30.512911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.512896 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-98x9m\"" Jul 14 09:26:30.512911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.512908 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 09:26:30.513167 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.513155 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 09:26:30.515952 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.515938 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 09:26:30.520299 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.520282 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 09:26:30.520381 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.520368 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" Jul 14 09:26:30.520489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.520473 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal"] Jul 14 09:26:30.530419 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.530381 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 09:21:29 +0000 UTC" deadline="2028-02-10 01:14:07.10477454 +0000 UTC" Jul 14 09:26:30.530419 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.530415 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13815h47m36.574361138s" Jul 14 09:26:30.544704 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.544683 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal"] Jul 14 09:26:30.545164 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.545151 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 09:26:30.557079 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.557061 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jvhdz" Jul 14 09:26:30.570145 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.570124 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jvhdz" Jul 14 09:26:30.582406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.582378 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 09:26:30.588641 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.588736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-kubelet\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.588736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-run-netns\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.588736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5debd252-3f9b-4896-877b-c63c3c208666-iptables-alerter-script\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.588736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588711 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-var-lib-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.588736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-ovn\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.588736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-cni-netd\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-etc-kubernetes\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-host\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588805 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnk2j\" (UniqueName: \"kubernetes.io/projected/5debd252-3f9b-4896-877b-c63c3c208666-kube-api-access-xnk2j\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-node-log\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588897 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-host\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovnkube-script-lib\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.588979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-os-release\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589004 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de28e022-2681-4080-9ca8-02d15fe6e813-multus-daemon-config\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa319f90-fab6-446b-b19f-cf24a7b53761-host\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-systemd-units\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589166 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-slash\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589190 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-systemd\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-etc-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589236 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovnkube-config\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovn-node-metrics-cert\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589299 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-lib-modules\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589331 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-cni-bin\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589348 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-system-cni-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589427 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-lib-modules\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589427 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-device-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysctl-d\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-systemd\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-device-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589479 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-env-overrides\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-systemd\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-cni-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589584 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-kubelet\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589598 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysctl-d\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fjtw\" (UniqueName: \"kubernetes.io/projected/de28e022-2681-4080-9ca8-02d15fe6e813-kube-api-access-8fjtw\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-cni-bin\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589653 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-cni-multus\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-registration-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589716 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-registration-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59d7060b-7f01-4a2f-9e55-503176c06e81-hosts-file\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.589829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-sys\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589786 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59d7060b-7f01-4a2f-9e55-503176c06e81-hosts-file\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589813 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b700b7f3-3775-43aa-941d-b3d90984957c-agent-certs\") pod \"konnectivity-agent-brzt6\" (UID: \"b700b7f3-3775-43aa-941d-b3d90984957c\") " pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-log-socket\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d7060b-7f01-4a2f-9e55-503176c06e81-tmp-dir\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-cnibin\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589938 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-os-release\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589912 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-sys\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.589952 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589970 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-cnibin\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590005 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-os-release\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.589962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590172 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d7060b-7f01-4a2f-9e55-503176c06e81-tmp-dir\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.590025 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:31.090005211 +0000 UTC m=+2.027855682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590511 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-var-lib-kubelet\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.590612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-etc-selinux\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590577 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysconfig\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-kubernetes\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590638 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5debd252-3f9b-4896-877b-c63c3c208666-host-slash\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590644 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-etc-selinux\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590646 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-var-lib-kubelet\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m2xg\" (UniqueName: \"kubernetes.io/projected/59d7060b-7f01-4a2f-9e55-503176c06e81-kube-api-access-2m2xg\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590662 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysconfig\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b6ms\" (UniqueName: \"kubernetes.io/projected/61a507c5-2f51-4ba3-9866-6e7397b160f4-kube-api-access-8b6ms\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-kubernetes\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b700b7f3-3775-43aa-941d-b3d90984957c-konnectivity-ca\") pod \"konnectivity-agent-brzt6\" (UID: \"b700b7f3-3775-43aa-941d-b3d90984957c\") " pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5pjn\" (UniqueName: \"kubernetes.io/projected/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-kube-api-access-g5pjn\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-conf-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-multus-certs\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.591518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-cnibin\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-system-cni-dir\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysctl-conf\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-tmp\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590974 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61a507c5-2f51-4ba3-9866-6e7397b160f4-system-cni-dir\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.590989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-socket-dir-parent\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591016 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-run\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591033 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591054 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de28e022-2681-4080-9ca8-02d15fe6e813-cni-binary-copy\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-socket-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591099 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-sysctl-conf\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-modprobe-d\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-run\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa319f90-fab6-446b-b19f-cf24a7b53761-serviceca\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.592152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591166 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54wh8\" (UniqueName: \"kubernetes.io/projected/fa319f90-fab6-446b-b19f-cf24a7b53761-kube-api-access-54wh8\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591190 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-modprobe-d\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-k8s-cni-cncf-io\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591235 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-socket-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591258 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-netns\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-hostroot\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591290 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61a507c5-2f51-4ba3-9866-6e7397b160f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591281 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8g2l\" (UniqueName: \"kubernetes.io/projected/36e51c26-bfe7-4fa9-ae78-3703c1e59027-kube-api-access-c8g2l\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-tuned\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srgrb\" (UniqueName: \"kubernetes.io/projected/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-kube-api-access-srgrb\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcd6f\" (UniqueName: \"kubernetes.io/projected/1bcf77f5-947b-4432-b408-425e3e0612a6-kube-api-access-vcd6f\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-sys-fs\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.591496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-kubelet-config\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.592128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-dbus\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.592229 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-sys-fs\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.592975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.592451 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36e51c26-bfe7-4fa9-ae78-3703c1e59027-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.594866 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.594836 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-tmp\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.595591 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.595573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-etc-tuned\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.595732 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.595718 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:30.595798 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.595736 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:30.595798 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.595763 2579 projected.go:196] Error preparing data for projected volume kube-api-access-5p8cl for pod openshift-network-diagnostics/network-check-target-8tlqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:30.595887 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.595829 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl podName:fc76ae33-b78a-436e-b22d-1810331bb41c nodeName:}" failed. No retries permitted until 2026-07-14 09:26:31.095812047 +0000 UTC m=+2.033662499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5p8cl" (UniqueName: "kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl") pod "network-check-target-8tlqr" (UID: "fc76ae33-b78a-436e-b22d-1810331bb41c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:30.600480 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.600453 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b6ms\" (UniqueName: \"kubernetes.io/projected/61a507c5-2f51-4ba3-9866-6e7397b160f4-kube-api-access-8b6ms\") pod \"multus-additional-cni-plugins-2g8sm\" (UID: \"61a507c5-2f51-4ba3-9866-6e7397b160f4\") " pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.600737 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.600717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m2xg\" (UniqueName: \"kubernetes.io/projected/59d7060b-7f01-4a2f-9e55-503176c06e81-kube-api-access-2m2xg\") pod \"node-resolver-zxfnl\" (UID: \"59d7060b-7f01-4a2f-9e55-503176c06e81\") " pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.601612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.601593 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srgrb\" (UniqueName: \"kubernetes.io/projected/8e99f04e-1d0e-4c93-802c-25e2a44ffe89-kube-api-access-srgrb\") pod \"tuned-4kwn8\" (UID: \"8e99f04e-1d0e-4c93-802c-25e2a44ffe89\") " pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.601709 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.601675 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8g2l\" (UniqueName: \"kubernetes.io/projected/36e51c26-bfe7-4fa9-ae78-3703c1e59027-kube-api-access-c8g2l\") pod \"aws-ebs-csi-driver-node-2qr5l\" (UID: \"36e51c26-bfe7-4fa9-ae78-3703c1e59027\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.601792 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.601769 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcd6f\" (UniqueName: \"kubernetes.io/projected/1bcf77f5-947b-4432-b408-425e3e0612a6-kube-api-access-vcd6f\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:30.692791 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692755 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovnkube-script-lib\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.692791 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-os-release\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de28e022-2681-4080-9ca8-02d15fe6e813-multus-daemon-config\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692841 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa319f90-fab6-446b-b19f-cf24a7b53761-host\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692856 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-systemd-units\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-slash\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692880 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-os-release\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692888 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-systemd\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692929 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa319f90-fab6-446b-b19f-cf24a7b53761-host\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692939 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-systemd\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692946 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-slash\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-etc-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692925 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-systemd-units\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.692986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovnkube-config\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693015 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-etc-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovn-node-metrics-cert\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-cni-bin\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693112 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-system-cni-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-env-overrides\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-cni-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693184 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-cni-bin\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-kubelet\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693219 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-kubelet\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fjtw\" (UniqueName: \"kubernetes.io/projected/de28e022-2681-4080-9ca8-02d15fe6e813-kube-api-access-8fjtw\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693225 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-system-cni-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693249 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-cni-bin\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-cni-multus\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b700b7f3-3775-43aa-941d-b3d90984957c-agent-certs\") pod \"konnectivity-agent-brzt6\" (UID: \"b700b7f3-3775-43aa-941d-b3d90984957c\") " pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693324 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovnkube-script-lib\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693328 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-log-socket\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693359 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-log-socket\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5debd252-3f9b-4896-877b-c63c3c208666-host-slash\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b700b7f3-3775-43aa-941d-b3d90984957c-konnectivity-ca\") pod \"konnectivity-agent-brzt6\" (UID: \"b700b7f3-3775-43aa-941d-b3d90984957c\") " pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.693735 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5pjn\" (UniqueName: \"kubernetes.io/projected/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-kube-api-access-g5pjn\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693485 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-cni-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693509 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de28e022-2681-4080-9ca8-02d15fe6e813-multus-daemon-config\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693533 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-cni-bin\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693543 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovnkube-config\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-conf-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-multus-certs\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-env-overrides\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-cnibin\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693648 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-conf-dir\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-socket-dir-parent\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-multus-certs\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693710 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-cnibin\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de28e022-2681-4080-9ca8-02d15fe6e813-cni-binary-copy\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693741 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.694581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693789 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-multus-socket-dir-parent\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa319f90-fab6-446b-b19f-cf24a7b53761-serviceca\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693843 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-var-lib-cni-multus\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54wh8\" (UniqueName: \"kubernetes.io/projected/fa319f90-fab6-446b-b19f-cf24a7b53761-kube-api-access-54wh8\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-k8s-cni-cncf-io\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-netns\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-hostroot\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-netns\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-kubelet-config\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-dbus\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-kubelet-config\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694057 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-hostroot\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-kubelet\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694086 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-kubelet\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-run-netns\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5debd252-3f9b-4896-877b-c63c3c208666-iptables-alerter-script\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de28e022-2681-4080-9ca8-02d15fe6e813-cni-binary-copy\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694129 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-dbus\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.693931 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-host-run-k8s-cni-cncf-io\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-var-lib-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.694167 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694185 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-run-netns\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694250 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-var-lib-openvswitch\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694279 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5debd252-3f9b-4896-877b-c63c3c208666-host-slash\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-ovn\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.694298 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret podName:bb6e4a21-f4f3-4033-bc5e-47f28e04b010 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:31.19428044 +0000 UTC m=+2.132130910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret") pod "global-pull-secret-syncer-8q6mb" (UID: "bb6e4a21-f4f3-4033-bc5e-47f28e04b010") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694331 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-run-ovn\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-cni-netd\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694414 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-etc-kubernetes\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnk2j\" (UniqueName: \"kubernetes.io/projected/5debd252-3f9b-4896-877b-c63c3c208666-kube-api-access-xnk2j\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694454 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-cni-netd\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-node-log\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de28e022-2681-4080-9ca8-02d15fe6e813-etc-kubernetes\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694516 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-node-log\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.695846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.696429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694550 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.696429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694686 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5debd252-3f9b-4896-877b-c63c3c208666-iptables-alerter-script\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.696429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b700b7f3-3775-43aa-941d-b3d90984957c-konnectivity-ca\") pod \"konnectivity-agent-brzt6\" (UID: \"b700b7f3-3775-43aa-941d-b3d90984957c\") " pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.696429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.694952 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa319f90-fab6-446b-b19f-cf24a7b53761-serviceca\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.696429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.695782 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-ovn-node-metrics-cert\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.696588 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.696468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b700b7f3-3775-43aa-941d-b3d90984957c-agent-certs\") pod \"konnectivity-agent-brzt6\" (UID: \"b700b7f3-3775-43aa-941d-b3d90984957c\") " pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.703245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.703223 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54wh8\" (UniqueName: \"kubernetes.io/projected/fa319f90-fab6-446b-b19f-cf24a7b53761-kube-api-access-54wh8\") pod \"node-ca-cglm5\" (UID: \"fa319f90-fab6-446b-b19f-cf24a7b53761\") " pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.703496 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.703478 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnk2j\" (UniqueName: \"kubernetes.io/projected/5debd252-3f9b-4896-877b-c63c3c208666-kube-api-access-xnk2j\") pod \"iptables-alerter-tjqrx\" (UID: \"5debd252-3f9b-4896-877b-c63c3c208666\") " pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.704601 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.704575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fjtw\" (UniqueName: \"kubernetes.io/projected/de28e022-2681-4080-9ca8-02d15fe6e813-kube-api-access-8fjtw\") pod \"multus-2mxt2\" (UID: \"de28e022-2681-4080-9ca8-02d15fe6e813\") " pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.705096 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.705079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5pjn\" (UniqueName: \"kubernetes.io/projected/bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6-kube-api-access-g5pjn\") pod \"ovnkube-node-4bjh8\" (UID: \"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.708395 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.708377 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:30.753639 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.753609 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06089b923710f2d2feae9e9599b422d3.slice/crio-83d3e6c354574d06e698d211592efd71f0035dacd6d5a4a7a133c2dec4318a7a WatchSource:0}: Error finding container 83d3e6c354574d06e698d211592efd71f0035dacd6d5a4a7a133c2dec4318a7a: Status 404 returned error can't find the container with id 83d3e6c354574d06e698d211592efd71f0035dacd6d5a4a7a133c2dec4318a7a Jul 14 09:26:30.753903 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.753884 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edd94a745f603ae25b0ce87a74b4849.slice/crio-1bc8cca7305abd2c0b254d3f32bed37dfe4e83d87848b24507556198456de564 WatchSource:0}: Error finding container 1bc8cca7305abd2c0b254d3f32bed37dfe4e83d87848b24507556198456de564: Status 404 returned error can't find the container with id 1bc8cca7305abd2c0b254d3f32bed37dfe4e83d87848b24507556198456de564 Jul 14 09:26:30.760151 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.760133 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 09:26:30.808610 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.808571 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" Jul 14 09:26:30.818399 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.818369 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e51c26_bfe7_4fa9_ae78_3703c1e59027.slice/crio-7c36b301b4307adb8e0d5f20b3c8276bc8bf60019473a61eedc81e3a531045ec WatchSource:0}: Error finding container 7c36b301b4307adb8e0d5f20b3c8276bc8bf60019473a61eedc81e3a531045ec: Status 404 returned error can't find the container with id 7c36b301b4307adb8e0d5f20b3c8276bc8bf60019473a61eedc81e3a531045ec Jul 14 09:26:30.822596 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.822576 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zxfnl" Jul 14 09:26:30.829664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.829645 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:30.831148 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.831127 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d7060b_7f01_4a2f_9e55_503176c06e81.slice/crio-5ad3c300eb375d049d86d605b3bd7bfd9dd26142d1fd11b86d51714100876944 WatchSource:0}: Error finding container 5ad3c300eb375d049d86d605b3bd7bfd9dd26142d1fd11b86d51714100876944: Status 404 returned error can't find the container with id 5ad3c300eb375d049d86d605b3bd7bfd9dd26142d1fd11b86d51714100876944 Jul 14 09:26:30.837613 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.837564 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" Jul 14 09:26:30.845992 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.845967 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a507c5_2f51_4ba3_9866_6e7397b160f4.slice/crio-86b36ad23142c19a6a85fc28cc5c681630022d8fb12ad4f064a0611afacd68ab WatchSource:0}: Error finding container 86b36ad23142c19a6a85fc28cc5c681630022d8fb12ad4f064a0611afacd68ab: Status 404 returned error can't find the container with id 86b36ad23142c19a6a85fc28cc5c681630022d8fb12ad4f064a0611afacd68ab Jul 14 09:26:30.852372 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.852348 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" Jul 14 09:26:30.860178 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.860149 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e99f04e_1d0e_4c93_802c_25e2a44ffe89.slice/crio-9deda5d923d18e9d194c3243ca348f1e1b1c0c111f9bb74f5f293ec4602756f6 WatchSource:0}: Error finding container 9deda5d923d18e9d194c3243ca348f1e1b1c0c111f9bb74f5f293ec4602756f6: Status 404 returned error can't find the container with id 9deda5d923d18e9d194c3243ca348f1e1b1c0c111f9bb74f5f293ec4602756f6 Jul 14 09:26:30.868610 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.868587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tjqrx" Jul 14 09:26:30.869691 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.869674 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:30.879203 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.879166 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5debd252_3f9b_4896_877b_c63c3c208666.slice/crio-d937f3f2fd1006587eb299717162dc4d5f0b3c45b9744d252d0d48b97e423f97 WatchSource:0}: Error finding container d937f3f2fd1006587eb299717162dc4d5f0b3c45b9744d252d0d48b97e423f97: Status 404 returned error can't find the container with id d937f3f2fd1006587eb299717162dc4d5f0b3c45b9744d252d0d48b97e423f97 Jul 14 09:26:30.879671 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.879654 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb700b7f3_3775_43aa_941d_b3d90984957c.slice/crio-820c9adf55f9733a0f23299982606065f32a0cd74627852d796ebb2005d2885c WatchSource:0}: Error finding container 820c9adf55f9733a0f23299982606065f32a0cd74627852d796ebb2005d2885c: Status 404 returned error can't find the container with id 820c9adf55f9733a0f23299982606065f32a0cd74627852d796ebb2005d2885c Jul 14 09:26:30.902491 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.902458 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cglm5" Jul 14 09:26:30.912199 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.912172 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa319f90_fab6_446b_b19f_cf24a7b53761.slice/crio-3393c24ea67cc0ae78f4bac39250159e4e9727e222846da0aa6470720fba8f56 WatchSource:0}: Error finding container 3393c24ea67cc0ae78f4bac39250159e4e9727e222846da0aa6470720fba8f56: Status 404 returned error can't find the container with id 3393c24ea67cc0ae78f4bac39250159e4e9727e222846da0aa6470720fba8f56 Jul 14 09:26:30.924546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.924523 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:30.929296 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:30.929280 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2mxt2" Jul 14 09:26:30.933698 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.933673 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc29122_d1d9_4ec6_b3cf_fd4a2b76ebb6.slice/crio-2331aa813b787bc58151061a8868e7ea89dd2e75a31932ecedf8eabcf195712e WatchSource:0}: Error finding container 2331aa813b787bc58151061a8868e7ea89dd2e75a31932ecedf8eabcf195712e: Status 404 returned error can't find the container with id 2331aa813b787bc58151061a8868e7ea89dd2e75a31932ecedf8eabcf195712e Jul 14 09:26:30.937980 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:26:30.937951 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde28e022_2681_4080_9ca8_02d15fe6e813.slice/crio-35023bb484a42d674ad7b0c8ea9230312e38106cc06a93c344141022fc1a667d WatchSource:0}: Error finding container 35023bb484a42d674ad7b0c8ea9230312e38106cc06a93c344141022fc1a667d: Status 404 returned error can't find the container with id 35023bb484a42d674ad7b0c8ea9230312e38106cc06a93c344141022fc1a667d Jul 14 09:26:30.939656 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.939638 2579 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-2mxt2_openshift-multus(de28e022-2681-4080-9ca8-02d15fe6e813): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 14 09:26:30.939761 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:30.939668 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-2mxt2" podUID="de28e022-2681-4080-9ca8-02d15fe6e813" Jul 14 09:26:31.098699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.098616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:31.098699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.098665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:31.098905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.098779 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:31.098905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.098797 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:31.098905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.098818 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:31.098905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.098828 2579 projected.go:196] Error preparing data for projected volume kube-api-access-5p8cl for pod openshift-network-diagnostics/network-check-target-8tlqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:31.098905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.098831 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:32.098817042 +0000 UTC m=+3.036667490 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:31.098905 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.098869 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl podName:fc76ae33-b78a-436e-b22d-1810331bb41c nodeName:}" failed. No retries permitted until 2026-07-14 09:26:32.098858426 +0000 UTC m=+3.036708874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5p8cl" (UniqueName: "kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl") pod "network-check-target-8tlqr" (UID: "fc76ae33-b78a-436e-b22d-1810331bb41c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:31.199254 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.199219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:31.199416 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.199384 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:31.199460 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.199449 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret podName:bb6e4a21-f4f3-4033-bc5e-47f28e04b010 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:32.199434244 +0000 UTC m=+3.137284692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret") pod "global-pull-secret-syncer-8q6mb" (UID: "bb6e4a21-f4f3-4033-bc5e-47f28e04b010") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:31.574000 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.573900 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 09:21:30 +0000 UTC" deadline="2027-12-17 13:53:45.371172154 +0000 UTC" Jul 14 09:26:31.574000 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.573997 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12508h27m13.797179154s" Jul 14 09:26:31.631904 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.631876 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:31.632187 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.632019 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:31.632652 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.632626 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:31.632813 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.632727 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:31.652843 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.652785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cglm5" event={"ID":"fa319f90-fab6-446b-b19f-cf24a7b53761","Type":"ContainerStarted","Data":"3393c24ea67cc0ae78f4bac39250159e4e9727e222846da0aa6470720fba8f56"} Jul 14 09:26:31.679426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.679386 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-brzt6" event={"ID":"b700b7f3-3775-43aa-941d-b3d90984957c","Type":"ContainerStarted","Data":"820c9adf55f9733a0f23299982606065f32a0cd74627852d796ebb2005d2885c"} Jul 14 09:26:31.690919 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.690876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" event={"ID":"8e99f04e-1d0e-4c93-802c-25e2a44ffe89","Type":"ContainerStarted","Data":"9deda5d923d18e9d194c3243ca348f1e1b1c0c111f9bb74f5f293ec4602756f6"} Jul 14 09:26:31.695515 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.695424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerStarted","Data":"86b36ad23142c19a6a85fc28cc5c681630022d8fb12ad4f064a0611afacd68ab"} Jul 14 09:26:31.702932 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.702903 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" event={"ID":"36e51c26-bfe7-4fa9-ae78-3703c1e59027","Type":"ContainerStarted","Data":"7c36b301b4307adb8e0d5f20b3c8276bc8bf60019473a61eedc81e3a531045ec"} Jul 14 09:26:31.711227 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.711177 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" event={"ID":"1edd94a745f603ae25b0ce87a74b4849","Type":"ContainerStarted","Data":"1bc8cca7305abd2c0b254d3f32bed37dfe4e83d87848b24507556198456de564"} Jul 14 09:26:31.719409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.719372 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2mxt2" event={"ID":"de28e022-2681-4080-9ca8-02d15fe6e813","Type":"ContainerStarted","Data":"35023bb484a42d674ad7b0c8ea9230312e38106cc06a93c344141022fc1a667d"} Jul 14 09:26:31.725179 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:31.725110 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5228585713df6fe73e63d85efc7cb37f2a929792a1afb637a77bf56a4f2a40f9\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-2mxt2" podUID="de28e022-2681-4080-9ca8-02d15fe6e813" Jul 14 09:26:31.743236 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.742996 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"2331aa813b787bc58151061a8868e7ea89dd2e75a31932ecedf8eabcf195712e"} Jul 14 09:26:31.752659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.752625 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tjqrx" event={"ID":"5debd252-3f9b-4896-877b-c63c3c208666","Type":"ContainerStarted","Data":"d937f3f2fd1006587eb299717162dc4d5f0b3c45b9744d252d0d48b97e423f97"} Jul 14 09:26:31.761108 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.761076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zxfnl" event={"ID":"59d7060b-7f01-4a2f-9e55-503176c06e81","Type":"ContainerStarted","Data":"5ad3c300eb375d049d86d605b3bd7bfd9dd26142d1fd11b86d51714100876944"} Jul 14 09:26:31.767235 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.767205 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" event={"ID":"06089b923710f2d2feae9e9599b422d3","Type":"ContainerStarted","Data":"83d3e6c354574d06e698d211592efd71f0035dacd6d5a4a7a133c2dec4318a7a"} Jul 14 09:26:31.816961 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:31.816928 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:32.106216 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:32.106178 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:32.106406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:32.106275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:32.106486 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.106450 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:32.106486 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.106468 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:32.106486 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.106484 2579 projected.go:196] Error preparing data for projected volume kube-api-access-5p8cl for pod openshift-network-diagnostics/network-check-target-8tlqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:32.106637 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.106542 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl podName:fc76ae33-b78a-436e-b22d-1810331bb41c nodeName:}" failed. No retries permitted until 2026-07-14 09:26:34.106522138 +0000 UTC m=+5.044372588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5p8cl" (UniqueName: "kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl") pod "network-check-target-8tlqr" (UID: "fc76ae33-b78a-436e-b22d-1810331bb41c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:32.106925 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.106903 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:32.107005 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.106969 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:34.106952793 +0000 UTC m=+5.044803261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:32.207007 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:32.206956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:32.207199 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.207125 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:32.207199 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.207196 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret podName:bb6e4a21-f4f3-4033-bc5e-47f28e04b010 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:34.207168572 +0000 UTC m=+5.145019030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret") pod "global-pull-secret-syncer-8q6mb" (UID: "bb6e4a21-f4f3-4033-bc5e-47f28e04b010") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:32.575222 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:32.575162 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 09:21:30 +0000 UTC" deadline="2028-01-14 11:57:36.279500839 +0000 UTC" Jul 14 09:26:32.575222 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:32.575220 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13178h31m3.704285629s" Jul 14 09:26:32.630874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:32.630766 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:32.631087 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:32.630918 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:33.631883 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:33.631851 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:33.632339 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:33.631968 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:33.632409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:33.632365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:33.632460 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:33.632444 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:34.125416 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:34.125313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:34.125600 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.125507 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:34.125600 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.125532 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:34.125600 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.125545 2579 projected.go:196] Error preparing data for projected volume kube-api-access-5p8cl for pod openshift-network-diagnostics/network-check-target-8tlqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:34.125600 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:34.125449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:34.125600 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.125602 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl podName:fc76ae33-b78a-436e-b22d-1810331bb41c nodeName:}" failed. No retries permitted until 2026-07-14 09:26:38.125583895 +0000 UTC m=+9.063434361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5p8cl" (UniqueName: "kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl") pod "network-check-target-8tlqr" (UID: "fc76ae33-b78a-436e-b22d-1810331bb41c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:34.125888 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.125606 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:34.125888 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.125655 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:38.125641193 +0000 UTC m=+9.063491642 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:34.226775 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:34.226672 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:34.226941 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.226884 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:34.227006 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.226967 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret podName:bb6e4a21-f4f3-4033-bc5e-47f28e04b010 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:38.226946853 +0000 UTC m=+9.164797315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret") pod "global-pull-secret-syncer-8q6mb" (UID: "bb6e4a21-f4f3-4033-bc5e-47f28e04b010") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:34.630426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:34.630387 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:34.630626 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:34.630525 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:35.630538 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:35.630502 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:35.631013 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:35.630630 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:35.631110 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:35.631091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:35.631230 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:35.631210 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:36.630235 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:36.630195 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:36.630422 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:36.630330 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:37.630952 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:37.630920 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:37.631401 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:37.631053 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:37.631401 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:37.631100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:37.631401 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:37.631233 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:38.156809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:38.156551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:38.157015 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.156724 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:38.157015 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:38.156885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:38.157015 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.156964 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:46.15694111 +0000 UTC m=+17.094791572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:38.157196 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.157036 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:38.157196 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.157056 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:38.157196 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.157069 2579 projected.go:196] Error preparing data for projected volume kube-api-access-5p8cl for pod openshift-network-diagnostics/network-check-target-8tlqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:38.157196 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.157130 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl podName:fc76ae33-b78a-436e-b22d-1810331bb41c nodeName:}" failed. No retries permitted until 2026-07-14 09:26:46.157115209 +0000 UTC m=+17.094965672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5p8cl" (UniqueName: "kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl") pod "network-check-target-8tlqr" (UID: "fc76ae33-b78a-436e-b22d-1810331bb41c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:38.257980 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:38.257944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:38.258174 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.258107 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:38.258237 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.258181 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret podName:bb6e4a21-f4f3-4033-bc5e-47f28e04b010 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:46.258164024 +0000 UTC m=+17.196014477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret") pod "global-pull-secret-syncer-8q6mb" (UID: "bb6e4a21-f4f3-4033-bc5e-47f28e04b010") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:38.630509 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:38.630423 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:38.630661 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:38.630556 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:39.631932 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:39.631842 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:39.632339 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:39.631991 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:39.632339 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:39.632023 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:39.632339 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:39.632149 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:40.630913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:40.630877 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:40.631096 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:40.631009 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:41.631044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:41.631010 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:41.631494 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:41.631010 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:41.631494 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:41.631156 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:41.631494 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:41.631295 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:42.631257 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:42.631219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:42.631702 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:42.631350 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:43.630502 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:43.630469 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:43.630683 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:43.630469 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:43.630683 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:43.630621 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:43.630819 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:43.630676 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:44.630337 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:44.630296 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:44.630815 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:44.630545 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:45.630556 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:45.630520 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:45.630940 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:45.630634 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:45.630940 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:45.630683 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:45.630940 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:45.630792 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.217855 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.217957 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.218111 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.218130 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.218143 2579 projected.go:196] Error preparing data for projected volume kube-api-access-5p8cl for pod openshift-network-diagnostics/network-check-target-8tlqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.218203 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl podName:fc76ae33-b78a-436e-b22d-1810331bb41c nodeName:}" failed. No retries permitted until 2026-07-14 09:27:02.218184397 +0000 UTC m=+33.156034848 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5p8cl" (UniqueName: "kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl") pod "network-check-target-8tlqr" (UID: "fc76ae33-b78a-436e-b22d-1810331bb41c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.218274 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:46.218695 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.218309 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:02.218298087 +0000 UTC m=+33.156148537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:46.318889 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.318853 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:46.319098 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.319071 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:46.319167 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.319145 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret podName:bb6e4a21-f4f3-4033-bc5e-47f28e04b010 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:02.31912579 +0000 UTC m=+33.256976251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret") pod "global-pull-secret-syncer-8q6mb" (UID: "bb6e4a21-f4f3-4033-bc5e-47f28e04b010") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:46.631269 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.630910 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:46.632168 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:46.631377 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:46.811420 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.811152 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cglm5" event={"ID":"fa319f90-fab6-446b-b19f-cf24a7b53761","Type":"ContainerStarted","Data":"e81c52fd5d6244bcd975754e217cb1ebd54411b2f9c90115a896fdb5fb835418"} Jul 14 09:26:46.812965 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.812937 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-brzt6" event={"ID":"b700b7f3-3775-43aa-941d-b3d90984957c","Type":"ContainerStarted","Data":"45cdbc65aad8ffe51997409f8fa610845ee1c0f3dc798b6efa6231d7adcdaf66"} Jul 14 09:26:46.814704 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.814683 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" event={"ID":"8e99f04e-1d0e-4c93-802c-25e2a44ffe89","Type":"ContainerStarted","Data":"280bb5e65af79f1b1114044c919ae8fb14b65e5f4344b644c90ea06e9e38a0f6"} Jul 14 09:26:46.816523 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.816497 2579 generic.go:356] "Generic (PLEG): container finished" podID="61a507c5-2f51-4ba3-9866-6e7397b160f4" containerID="cfa7c44e1e548c6ddd9a6b8a88a06f601307ac624df18dbf22de45757e694b69" exitCode=0 Jul 14 09:26:46.816630 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.816580 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerDied","Data":"cfa7c44e1e548c6ddd9a6b8a88a06f601307ac624df18dbf22de45757e694b69"} Jul 14 09:26:46.818473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.818432 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" event={"ID":"36e51c26-bfe7-4fa9-ae78-3703c1e59027","Type":"ContainerStarted","Data":"9b4121666d4572ec008637e7808a265e90f1adb47ad131b86cc6adfea50776ae"} Jul 14 09:26:46.820341 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.820315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" event={"ID":"1edd94a745f603ae25b0ce87a74b4849","Type":"ContainerStarted","Data":"609aa798641c09c323dd0ce351bdc2bd1ef92944ee3c405a361ef08a38965fc2"} Jul 14 09:26:46.823985 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.823953 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"962e2cdf18376d14acea42d51e30a511ee1b0eca76d2302390b6a0acac7d8bf8"} Jul 14 09:26:46.824089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.824039 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"8ebe117ae871a38e05da8f3f78bab23ebf26c4d4bee7ef00a03320e38873b67e"} Jul 14 09:26:46.824156 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.824089 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"0be7273133b64c5bbd0740559e739be5c30ec8ad10c193a817f399ebb259832f"} Jul 14 09:26:46.824156 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.824150 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"cce143b9fa3e27390e25cb26e70abf2cbe315e4ae0b57d84795c8135a831c51d"} Jul 14 09:26:46.824236 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.824200 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"fd4ed089f95b32826cba15a0b3d418f0ea49d583eeb2ae632c16f80110663efb"} Jul 14 09:26:46.824271 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.824248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"dd9d6ae5ca853b7fff6b8153806cf850a74f6077620dff967138e5e885d29816"} Jul 14 09:26:46.825624 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.825600 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zxfnl" event={"ID":"59d7060b-7f01-4a2f-9e55-503176c06e81","Type":"ContainerStarted","Data":"556a5af68d175a44b1c61c8dab40bb0a60927c18b818f92937265cefaa4467fe"} Jul 14 09:26:46.827434 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.827365 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cglm5" podStartSLOduration=6.55036516 podStartE2EDuration="16.827350281s" totalImagesPullingTime="10.276985121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:46.826873079 +0000 UTC m=+17.764723573" watchObservedRunningTime="2026-07-14 09:26:46.827350281 +0000 UTC m=+17.765200749" Jul 14 09:26:46.827775 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.827731 2579 generic.go:356] "Generic (PLEG): container finished" podID="06089b923710f2d2feae9e9599b422d3" containerID="375b802a8eabd85a9d686a14321ec231f06017b67807794bebe6be1d1a2ae444" exitCode=0 Jul 14 09:26:46.827867 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.827790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" event={"ID":"06089b923710f2d2feae9e9599b422d3","Type":"ContainerDied","Data":"375b802a8eabd85a9d686a14321ec231f06017b67807794bebe6be1d1a2ae444"} Jul 14 09:26:46.854452 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.854281 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-42.ec2.internal" podStartSLOduration=16.85425919 podStartE2EDuration="16.85425919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:46.83972138 +0000 UTC m=+17.777571849" watchObservedRunningTime="2026-07-14 09:26:46.85425919 +0000 UTC m=+17.792109661" Jul 14 09:26:46.854609 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.854576 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zxfnl" podStartSLOduration=2.90612936 podStartE2EDuration="17.854567307s" totalImagesPullingTime="14.948437947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:46.854132813 +0000 UTC m=+17.791983283" watchObservedRunningTime="2026-07-14 09:26:46.854567307 +0000 UTC m=+17.792417778" Jul 14 09:26:46.893025 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.892963 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4kwn8" podStartSLOduration=2.946305274 podStartE2EDuration="17.892942351s" totalImagesPullingTime="14.946637077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:46.892376142 +0000 UTC m=+17.830226613" watchObservedRunningTime="2026-07-14 09:26:46.892942351 +0000 UTC m=+17.830792824" Jul 14 09:26:46.906089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:46.906040 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-brzt6" podStartSLOduration=2.025857404 podStartE2EDuration="16.906024232s" totalImagesPullingTime="14.880166828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:46.905426304 +0000 UTC m=+17.843276776" watchObservedRunningTime="2026-07-14 09:26:46.906024232 +0000 UTC m=+17.843874702" Jul 14 09:26:47.517227 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.517199 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:47.518089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.517878 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:47.525762 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.525725 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 09:26:47.594505 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.594386 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T09:26:47.525763192Z","UUID":"6b30e6a5-eef7-4c2a-8e7e-87250e154cc0","Handler":null,"Name":"","Endpoint":""} Jul 14 09:26:47.596209 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.596187 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 09:26:47.596347 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.596231 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 09:26:47.630401 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.630367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:47.630586 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:47.630509 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:47.630880 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.630856 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:47.631010 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:47.630964 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:47.832519 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.832059 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tjqrx" event={"ID":"5debd252-3f9b-4896-877b-c63c3c208666","Type":"ContainerStarted","Data":"676b829f3a35a624e36c3189d311efaadbf08461fd22845e1e59a79c622eff03"} Jul 14 09:26:47.834441 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.834405 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" event={"ID":"06089b923710f2d2feae9e9599b422d3","Type":"ContainerStarted","Data":"fad6ecb217088ae2057d806a918f02a30ab17153db9c4ebe90b883290f6fab68"} Jul 14 09:26:47.837306 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.837277 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" event={"ID":"36e51c26-bfe7-4fa9-ae78-3703c1e59027","Type":"ContainerStarted","Data":"d7799063495ba89d55dab29fc2047c17a41db129f7d965ab73cd1b566d221ce0"} Jul 14 09:26:47.847467 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.847412 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tjqrx" podStartSLOduration=3.947482125 podStartE2EDuration="18.847391984s" totalImagesPullingTime="14.899909859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:47.846764917 +0000 UTC m=+18.784615380" watchObservedRunningTime="2026-07-14 09:26:47.847391984 +0000 UTC m=+18.785242456" Jul 14 09:26:47.861325 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:47.861275 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-42.ec2.internal" podStartSLOduration=17.861255346 podStartE2EDuration="17.861255346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:47.860897309 +0000 UTC m=+18.798747797" watchObservedRunningTime="2026-07-14 09:26:47.861255346 +0000 UTC m=+18.799105815" Jul 14 09:26:48.630825 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:48.630792 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:48.631018 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:48.630965 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:48.842562 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:48.842520 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" event={"ID":"36e51c26-bfe7-4fa9-ae78-3703c1e59027","Type":"ContainerStarted","Data":"f1367dccccdccd907676ce0f9ddc81c59f02933feaac8a26fb039fbf7e3f7772"} Jul 14 09:26:48.842975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:48.842536 2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 09:26:48.859011 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:48.858957 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2qr5l" podStartSLOduration=2.981510561 podStartE2EDuration="19.858936039s" totalImagesPullingTime="16.877425478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:48.858814205 +0000 UTC m=+19.796664675" watchObservedRunningTime="2026-07-14 09:26:48.858936039 +0000 UTC m=+19.796786511" Jul 14 09:26:49.631443 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:49.631405 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:49.631641 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:49.631514 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:49.631881 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:49.631859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:49.631979 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:49.631949 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:49.852892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:49.852758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"3caf271fb5313b0160224db4fbe3ed5a51c02338248acd969dc675bfc40e015a"} Jul 14 09:26:50.630191 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:50.630146 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:50.630363 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:50.630290 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:51.634694 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:51.634525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:51.635135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:51.634588 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:51.635135 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:51.634799 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:51.635135 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:51.634894 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:52.503342 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:52.503300 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:52.503521 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:52.503456 2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 09:26:52.503967 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:52.503937 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-brzt6" Jul 14 09:26:52.630802 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:52.630768 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:52.630974 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:52.630914 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:53.630863 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:53.630827 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:53.631484 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:53.630976 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:53.631484 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:53.631036 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:53.631484 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:53.631138 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:54.630293 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:54.630259 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:54.630466 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:54.630400 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:55.630326 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.630166 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:55.631025 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.630211 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:55.631025 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:55.630422 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:55.631025 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:55.630493 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:55.867652 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.867544 2579 generic.go:356] "Generic (PLEG): container finished" podID="61a507c5-2f51-4ba3-9866-6e7397b160f4" containerID="d9554f670797160e2bf5214414f60b7fe7840ce27e5e3b0a67aee4fc86a078fe" exitCode=0 Jul 14 09:26:55.867652 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.867634 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerDied","Data":"d9554f670797160e2bf5214414f60b7fe7840ce27e5e3b0a67aee4fc86a078fe"} Jul 14 09:26:55.869369 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.869349 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2mxt2" event={"ID":"de28e022-2681-4080-9ca8-02d15fe6e813","Type":"ContainerStarted","Data":"77faad4d18d21979a617802d3d22ffe1e9a91689f16e605fea80b30db078b0b3"} Jul 14 09:26:55.873283 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.873262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" event={"ID":"bdc29122-d1d9-4ec6-b3cf-fd4a2b76ebb6","Type":"ContainerStarted","Data":"7027ab32eddfa3fd40980700669bb1dab06a5a941aaa8e79edbc468e38877b1e"} Jul 14 09:26:55.873553 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.873530 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:55.873640 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.873577 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:55.873640 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.873615 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:55.888988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.888959 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:55.889100 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.889080 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:26:55.914842 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.914788 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" podStartSLOduration=10.606762161 podStartE2EDuration="25.914769409s" totalImagesPullingTime="15.308007248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:26:55.91463925 +0000 UTC m=+26.852489719" watchObservedRunningTime="2026-07-14 09:26:55.914769409 +0000 UTC m=+26.852619871" Jul 14 09:26:55.931131 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:55.931090 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2mxt2" podStartSLOduration=1.241191848 podStartE2EDuration="25.931075538s" totalImagesPullingTime="24.68988369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-14 09:26:55.930977478 +0000 UTC m=+26.868827949" watchObservedRunningTime="2026-07-14 09:26:55.931075538 +0000 UTC m=+26.868926007" Jul 14 09:26:56.630549 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:56.630519 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:56.631030 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:56.630643 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:57.396945 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.396916 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8tlqr"] Jul 14 09:26:57.397136 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.397060 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:57.397201 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:57.397154 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:57.399880 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.399851 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8q6mb"] Jul 14 09:26:57.400337 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.400165 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:57.400337 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:57.400291 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:57.400545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.400527 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fpqzv"] Jul 14 09:26:57.400635 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.400622 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:57.400759 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:57.400724 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:57.880964 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.880938 2579 generic.go:356] "Generic (PLEG): container finished" podID="61a507c5-2f51-4ba3-9866-6e7397b160f4" containerID="f7870cb38c064053accc7429f54d0157beb5bb1683a591eb4338206f95f3ef13" exitCode=0 Jul 14 09:26:57.881360 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:57.881022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerDied","Data":"f7870cb38c064053accc7429f54d0157beb5bb1683a591eb4338206f95f3ef13"} Jul 14 09:26:59.632986 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:59.632731 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:26:59.633535 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:59.633100 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tlqr" podUID="fc76ae33-b78a-436e-b22d-1810331bb41c" Jul 14 09:26:59.633628 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:59.633607 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:26:59.633819 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:59.633795 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fpqzv" podUID="1bcf77f5-947b-4432-b408-425e3e0612a6" Jul 14 09:26:59.633889 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:59.633864 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:26:59.634007 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:26:59.633984 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8q6mb" podUID="bb6e4a21-f4f3-4033-bc5e-47f28e04b010" Jul 14 09:26:59.886659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:59.886625 2579 generic.go:356] "Generic (PLEG): container finished" podID="61a507c5-2f51-4ba3-9866-6e7397b160f4" containerID="6873394a5f156feb516474b9a895ec1c057f9a4c53e5153da3b21200872f9bf0" exitCode=0 Jul 14 09:26:59.886841 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:26:59.886682 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerDied","Data":"6873394a5f156feb516474b9a895ec1c057f9a4c53e5153da3b21200872f9bf0"} Jul 14 09:27:01.387476 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.387405 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-42.ec2.internal" event="NodeReady" Jul 14 09:27:01.388012 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.387566 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 09:27:01.424821 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.424796 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76fc449dc4-ft7gq"] Jul 14 09:27:01.427384 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.427366 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.430699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.430675 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 09:27:01.430851 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.430765 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 09:27:01.430851 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.430787 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-prphk\"" Jul 14 09:27:01.430972 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.430912 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 09:27:01.446071 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.446043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76fc449dc4-ft7gq"] Jul 14 09:27:01.446977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.446954 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 09:27:01.452406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.452385 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j9jpq"] Jul 14 09:27:01.455873 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.454928 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.458089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.458060 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fxlg4\"" Jul 14 09:27:01.458089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.458082 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 09:27:01.458460 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.458445 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 09:27:01.459862 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.459840 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 09:27:01.460728 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.460707 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5vbvj"] Jul 14 09:27:01.462672 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.462652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.464724 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.464704 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 09:27:01.464992 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.464975 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jncc8\"" Jul 14 09:27:01.465316 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.465300 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 09:27:01.474804 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.474785 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j9jpq"] Jul 14 09:27:01.488963 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.488904 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5vbvj"] Jul 14 09:27:01.531663 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531634 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/259e96dd-ba4d-4586-8680-ffc219915b18-cert\") pod \"ingress-canary-j9jpq\" (UID: \"259e96dd-ba4d-4586-8680-ffc219915b18\") " pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.531663 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531672 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-registry-certificates\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.531914 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-bound-sa-token\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.531914 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531718 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg2q4\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-kube-api-access-vg2q4\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.531914 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce209c62-cc38-41b6-8021-d09ba761e072-tmp-dir\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.531914 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-image-registry-private-configuration\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.531914 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531906 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-trusted-ca\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.532132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce209c62-cc38-41b6-8021-d09ba761e072-config-volume\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.532132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.531980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkdww\" (UniqueName: \"kubernetes.io/projected/259e96dd-ba4d-4586-8680-ffc219915b18-kube-api-access-vkdww\") pod \"ingress-canary-j9jpq\" (UID: \"259e96dd-ba4d-4586-8680-ffc219915b18\") " pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.532132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.532014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be49dc80-58a0-48ae-851d-42ac05324d71-ca-trust-extracted\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.532132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.532045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.532132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.532097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7qn9\" (UniqueName: \"kubernetes.io/projected/ce209c62-cc38-41b6-8021-d09ba761e072-kube-api-access-x7qn9\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.532132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.532129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.532362 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.532152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-installation-pull-secrets\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.630506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.630237 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:27:01.630506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.630261 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:27:01.630506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.630236 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:27:01.630506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.630672 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:01.632546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632518 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/259e96dd-ba4d-4586-8680-ffc219915b18-cert\") pod \"ingress-canary-j9jpq\" (UID: \"259e96dd-ba4d-4586-8680-ffc219915b18\") " pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.632666 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-registry-certificates\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632666 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-bound-sa-token\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632666 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632603 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg2q4\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-kube-api-access-vg2q4\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632666 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce209c62-cc38-41b6-8021-d09ba761e072-tmp-dir\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.632666 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-image-registry-private-configuration\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-trusted-ca\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce209c62-cc38-41b6-8021-d09ba761e072-config-volume\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkdww\" (UniqueName: \"kubernetes.io/projected/259e96dd-ba4d-4586-8680-ffc219915b18-kube-api-access-vkdww\") pod \"ingress-canary-j9jpq\" (UID: \"259e96dd-ba4d-4586-8680-ffc219915b18\") " pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be49dc80-58a0-48ae-851d-42ac05324d71-ca-trust-extracted\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632824 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7qn9\" (UniqueName: \"kubernetes.io/projected/ce209c62-cc38-41b6-8021-d09ba761e072-kube-api-access-x7qn9\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-installation-pull-secrets\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.632948 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.632917 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frh48\"" Jul 14 09:27:01.633384 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633112 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 09:27:01.633384 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633218 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 09:27:01.633384 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633237 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-registry-certificates\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.633527 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633430 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 09:27:01.633603 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:01.633577 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:01.633653 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:01.633643 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls podName:ce209c62-cc38-41b6-8021-d09ba761e072 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:02.133625743 +0000 UTC m=+33.071476194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls") pod "dns-default-5vbvj" (UID: "ce209c62-cc38-41b6-8021-d09ba761e072") : secret "dns-default-metrics-tls" not found Jul 14 09:27:01.633715 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633647 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce209c62-cc38-41b6-8021-d09ba761e072-tmp-dir\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.633715 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633705 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ks9qb\"" Jul 14 09:27:01.633896 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce209c62-cc38-41b6-8021-d09ba761e072-config-volume\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.633960 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.633941 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 09:27:01.634775 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:01.634031 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:01.634775 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:01.634047 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fc449dc4-ft7gq: secret "image-registry-tls" not found Jul 14 09:27:01.634775 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:01.634091 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls podName:be49dc80-58a0-48ae-851d-42ac05324d71 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:02.134074182 +0000 UTC m=+33.071924649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls") pod "image-registry-76fc449dc4-ft7gq" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71") : secret "image-registry-tls" not found Jul 14 09:27:01.634775 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.634138 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be49dc80-58a0-48ae-851d-42ac05324d71-ca-trust-extracted\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.635387 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.635364 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-trusted-ca\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.637738 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.637684 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-installation-pull-secrets\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.638125 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.638103 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-image-registry-private-configuration\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.638197 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.638173 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/259e96dd-ba4d-4586-8680-ffc219915b18-cert\") pod \"ingress-canary-j9jpq\" (UID: \"259e96dd-ba4d-4586-8680-ffc219915b18\") " pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.644427 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.644397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-bound-sa-token\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.645379 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.645358 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7qn9\" (UniqueName: \"kubernetes.io/projected/ce209c62-cc38-41b6-8021-d09ba761e072-kube-api-access-x7qn9\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:01.645693 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.645672 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg2q4\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-kube-api-access-vg2q4\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:01.645831 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.645810 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkdww\" (UniqueName: \"kubernetes.io/projected/259e96dd-ba4d-4586-8680-ffc219915b18-kube-api-access-vkdww\") pod \"ingress-canary-j9jpq\" (UID: \"259e96dd-ba4d-4586-8680-ffc219915b18\") " pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.725928 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.725735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:01.765680 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.765650 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j9jpq" Jul 14 09:27:01.920315 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:01.919975 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j9jpq"] Jul 14 09:27:01.924718 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:01.924685 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259e96dd_ba4d_4586_8680_ffc219915b18.slice/crio-9ea5da90f687c1695914daa40a7feab64a21d2cb119bd1c896fbe03d9e6e7248 WatchSource:0}: Error finding container 9ea5da90f687c1695914daa40a7feab64a21d2cb119bd1c896fbe03d9e6e7248: Status 404 returned error can't find the container with id 9ea5da90f687c1695914daa40a7feab64a21d2cb119bd1c896fbe03d9e6e7248 Jul 14 09:27:02.136324 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.136285 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:02.136519 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.136336 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:02.136519 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.136448 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:02.136519 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.136515 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls podName:ce209c62-cc38-41b6-8021-d09ba761e072 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:03.136496006 +0000 UTC m=+34.074346457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls") pod "dns-default-5vbvj" (UID: "ce209c62-cc38-41b6-8021-d09ba761e072") : secret "dns-default-metrics-tls" not found Jul 14 09:27:02.136655 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.136530 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:02.136655 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.136555 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fc449dc4-ft7gq: secret "image-registry-tls" not found Jul 14 09:27:02.136655 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.136617 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls podName:be49dc80-58a0-48ae-851d-42ac05324d71 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:03.136597585 +0000 UTC m=+34.074448041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls") pod "image-registry-76fc449dc4-ft7gq" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71") : secret "image-registry-tls" not found Jul 14 09:27:02.237182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.237101 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:27:02.237182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.237165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:27:02.237401 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.237280 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 09:27:02.237401 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:02.237347 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:34.237329448 +0000 UTC m=+65.175179910 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : secret "metrics-daemon-secret" not found Jul 14 09:27:02.241012 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.240980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8cl\" (UniqueName: \"kubernetes.io/projected/fc76ae33-b78a-436e-b22d-1810331bb41c-kube-api-access-5p8cl\") pod \"network-check-target-8tlqr\" (UID: \"fc76ae33-b78a-436e-b22d-1810331bb41c\") " pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:27:02.258796 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.258762 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:27:02.337569 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.337539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:27:02.340469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.340445 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb6e4a21-f4f3-4033-bc5e-47f28e04b010-original-pull-secret\") pod \"global-pull-secret-syncer-8q6mb\" (UID: \"bb6e4a21-f4f3-4033-bc5e-47f28e04b010\") " pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:27:02.398582 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.398554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8tlqr"] Jul 14 09:27:02.402686 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:02.402660 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc76ae33_b78a_436e_b22d_1810331bb41c.slice/crio-ec5ff58bc6c1d236445da34e2dc4ef85d29a440d868ce348fd6e2c66decad619 WatchSource:0}: Error finding container ec5ff58bc6c1d236445da34e2dc4ef85d29a440d868ce348fd6e2c66decad619: Status 404 returned error can't find the container with id ec5ff58bc6c1d236445da34e2dc4ef85d29a440d868ce348fd6e2c66decad619 Jul 14 09:27:02.571344 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.571304 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8q6mb" Jul 14 09:27:02.895786 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.895677 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j9jpq" event={"ID":"259e96dd-ba4d-4586-8680-ffc219915b18","Type":"ContainerStarted","Data":"9ea5da90f687c1695914daa40a7feab64a21d2cb119bd1c896fbe03d9e6e7248"} Jul 14 09:27:02.896975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:02.896937 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8tlqr" event={"ID":"fc76ae33-b78a-436e-b22d-1810331bb41c","Type":"ContainerStarted","Data":"ec5ff58bc6c1d236445da34e2dc4ef85d29a440d868ce348fd6e2c66decad619"} Jul 14 09:27:03.144591 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:03.144547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:03.144785 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:03.144656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:03.144785 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:03.144718 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:03.144785 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:03.144739 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fc449dc4-ft7gq: secret "image-registry-tls" not found Jul 14 09:27:03.144918 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:03.144787 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:03.144918 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:03.144817 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls podName:be49dc80-58a0-48ae-851d-42ac05324d71 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:05.144797092 +0000 UTC m=+36.082647545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls") pod "image-registry-76fc449dc4-ft7gq" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71") : secret "image-registry-tls" not found Jul 14 09:27:03.144918 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:03.144839 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls podName:ce209c62-cc38-41b6-8021-d09ba761e072 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:05.144824138 +0000 UTC m=+36.082674605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls") pod "dns-default-5vbvj" (UID: "ce209c62-cc38-41b6-8021-d09ba761e072") : secret "dns-default-metrics-tls" not found Jul 14 09:27:03.518911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:03.518871 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:04.047611 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:04.047580 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8q6mb"] Jul 14 09:27:04.052020 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:04.051969 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb6e4a21_f4f3_4033_bc5e_47f28e04b010.slice/crio-8d0c73c7e972a699183df4072f875a3ba7007277d9b8244d2073a93b4c570b0a WatchSource:0}: Error finding container 8d0c73c7e972a699183df4072f875a3ba7007277d9b8244d2073a93b4c570b0a: Status 404 returned error can't find the container with id 8d0c73c7e972a699183df4072f875a3ba7007277d9b8244d2073a93b4c570b0a Jul 14 09:27:04.904848 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:04.904770 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8q6mb" event={"ID":"bb6e4a21-f4f3-4033-bc5e-47f28e04b010","Type":"ContainerStarted","Data":"8d0c73c7e972a699183df4072f875a3ba7007277d9b8244d2073a93b4c570b0a"} Jul 14 09:27:04.908734 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:04.908645 2579 generic.go:356] "Generic (PLEG): container finished" podID="61a507c5-2f51-4ba3-9866-6e7397b160f4" containerID="054eff97e78349781bded0da9eca39449256ac6f017900708f0ffbc13324750a" exitCode=0 Jul 14 09:27:04.908734 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:04.908690 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerDied","Data":"054eff97e78349781bded0da9eca39449256ac6f017900708f0ffbc13324750a"} Jul 14 09:27:05.161065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:05.160975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:05.161065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:05.161031 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:05.161284 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:05.161148 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:05.161284 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:05.161229 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls podName:ce209c62-cc38-41b6-8021-d09ba761e072 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:09.161206685 +0000 UTC m=+40.099057136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls") pod "dns-default-5vbvj" (UID: "ce209c62-cc38-41b6-8021-d09ba761e072") : secret "dns-default-metrics-tls" not found Jul 14 09:27:05.161284 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:05.161157 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:05.161284 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:05.161263 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fc449dc4-ft7gq: secret "image-registry-tls" not found Jul 14 09:27:05.161498 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:05.161308 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls podName:be49dc80-58a0-48ae-851d-42ac05324d71 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:09.161295387 +0000 UTC m=+40.099145839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls") pod "image-registry-76fc449dc4-ft7gq" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71") : secret "image-registry-tls" not found Jul 14 09:27:05.651001 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:05.650913 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:05.750758 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:05.750714 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:05.916187 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:05.916096 2579 generic.go:356] "Generic (PLEG): container finished" podID="61a507c5-2f51-4ba3-9866-6e7397b160f4" containerID="f74f4671609f61b287dfef1f72eee99a74a85157f44660accce0a37950ce2370" exitCode=0 Jul 14 09:27:05.916187 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:05.916170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerDied","Data":"f74f4671609f61b287dfef1f72eee99a74a85157f44660accce0a37950ce2370"} Jul 14 09:27:06.722461 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:06.722324 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:06.823259 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:06.823219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:06.922969 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:06.922933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:09.191586 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.191553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:09.191904 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.191598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:09.191904 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:09.191703 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:09.191904 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:09.191705 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:09.191904 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:09.191714 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fc449dc4-ft7gq: secret "image-registry-tls" not found Jul 14 09:27:09.191904 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:09.191799 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls podName:ce209c62-cc38-41b6-8021-d09ba761e072 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:17.191782453 +0000 UTC m=+48.129632905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls") pod "dns-default-5vbvj" (UID: "ce209c62-cc38-41b6-8021-d09ba761e072") : secret "dns-default-metrics-tls" not found Jul 14 09:27:09.191904 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:09.191863 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls podName:be49dc80-58a0-48ae-851d-42ac05324d71 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:17.191849788 +0000 UTC m=+48.129700237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls") pod "image-registry-76fc449dc4-ft7gq" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71") : secret "image-registry-tls" not found Jul 14 09:27:09.607631 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.607602 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:09.679288 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.679257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:09.929536 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.929456 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" event={"ID":"61a507c5-2f51-4ba3-9866-6e7397b160f4","Type":"ContainerStarted","Data":"3669aa39107098fdf0f86d2b6f40195622df17d48b7402c95fcd48cc9fbc7e2d"} Jul 14 09:27:09.931006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.930978 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8tlqr" event={"ID":"fc76ae33-b78a-436e-b22d-1810331bb41c","Type":"ContainerStarted","Data":"9ad88177fb1b78ef7ac3e5db707361f7497cda8368e364f4f7441e9360231e78"} Jul 14 09:27:09.931153 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.931031 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:27:09.932407 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.932381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j9jpq" event={"ID":"259e96dd-ba4d-4586-8680-ffc219915b18","Type":"ContainerStarted","Data":"02ccf0d338cdf4a131e1d2dad6190ae255a28b0ea3bdd3d89da5d805455f570f"} Jul 14 09:27:09.933806 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.933773 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8q6mb" event={"ID":"bb6e4a21-f4f3-4033-bc5e-47f28e04b010","Type":"ContainerStarted","Data":"9fe45dc806f393ff0a538cac6fe8f1de7014c11b9266903943be17b2b03599c9"} Jul 14 09:27:09.953784 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.953713 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2g8sm" podStartSLOduration=11.182578383 podStartE2EDuration="40.953700579s" totalImagesPullingTime="29.771122196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:09.952653823 +0000 UTC m=+40.890504294" watchObservedRunningTime="2026-07-14 09:27:09.953700579 +0000 UTC m=+40.891551048" Jul 14 09:27:09.967358 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.967312 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8tlqr" podStartSLOduration=33.814022113 podStartE2EDuration="40.967299557s" totalImagesPullingTime="7.153277444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:09.966703718 +0000 UTC m=+40.904554184" watchObservedRunningTime="2026-07-14 09:27:09.967299557 +0000 UTC m=+40.905150009" Jul 14 09:27:09.985109 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.985065 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j9jpq" podStartSLOduration=1.351986955 podStartE2EDuration="8.985051571s" totalImagesPullingTime="7.633064616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:09.984499885 +0000 UTC m=+40.922350354" watchObservedRunningTime="2026-07-14 09:27:09.985051571 +0000 UTC m=+40.922902041" Jul 14 09:27:09.998567 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:09.998518 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8q6mb" podStartSLOduration=34.480159638 podStartE2EDuration="39.998501403s" totalImagesPullingTime="5.518341765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:09.998203289 +0000 UTC m=+40.936053760" watchObservedRunningTime="2026-07-14 09:27:09.998501403 +0000 UTC m=+40.936351874" Jul 14 09:27:10.076361 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:10.076314 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:10.144413 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:10.144373 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:12.363806 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.363772 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44"] Jul 14 09:27:12.385141 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.385112 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc"] Jul 14 09:27:12.385276 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.385237 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" Jul 14 09:27:12.387605 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.387583 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5x4qj\"" Jul 14 09:27:12.405242 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.405219 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tkhdz"] Jul 14 09:27:12.405383 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.405356 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.407803 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.407775 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.407803 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.407782 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-csjkx\"" Jul 14 09:27:12.407990 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.407826 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.407990 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.407857 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 09:27:12.407990 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.407857 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 09:27:12.429722 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.429700 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.429722 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.429706 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44"] Jul 14 09:27:12.429927 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.429766 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc"] Jul 14 09:27:12.429927 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.429784 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tkhdz"] Jul 14 09:27:12.431907 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.431889 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.431999 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.431978 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.432262 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.432248 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 09:27:12.432349 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.432332 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 09:27:12.432409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.432371 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fpdcl\"" Jul 14 09:27:12.437311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.437291 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 09:27:12.461403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.461377 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8"] Jul 14 09:27:12.480405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.480377 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4"] Jul 14 09:27:12.480522 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.480495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" Jul 14 09:27:12.483456 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.483433 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.483820 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.483798 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.483909 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.483851 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sxbzd\"" Jul 14 09:27:12.500938 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.500911 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.501052 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.500912 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-x4gsp"] Jul 14 09:27:12.504128 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.504104 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.504256 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.504171 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 14 09:27:12.504256 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.504190 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.504256 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.504208 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zm2jz\"" Jul 14 09:27:12.504413 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.504279 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 14 09:27:12.513967 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.513949 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5e16cd1-ac7f-419f-a206-ee417e748911-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.514065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.513973 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cclv7\" (UniqueName: \"kubernetes.io/projected/a5e16cd1-ac7f-419f-a206-ee417e748911-kube-api-access-cclv7\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.514065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514005 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5e16cd1-ac7f-419f-a206-ee417e748911-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.514065 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514027 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5e16cd1-ac7f-419f-a206-ee417e748911-serving-cert\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.514196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7e096b9-8d20-49c2-b4ca-22041fb503fc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.514196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514101 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxk46\" (UniqueName: \"kubernetes.io/projected/a7e096b9-8d20-49c2-b4ca-22041fb503fc-kube-api-access-xxk46\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.514196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz65b\" (UniqueName: \"kubernetes.io/projected/ba77dcda-507d-47ee-94ed-e57334e8e4b7-kube-api-access-gz65b\") pod \"network-check-source-869dcfcdcd-2jw44\" (UID: \"ba77dcda-507d-47ee-94ed-e57334e8e4b7\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" Jul 14 09:27:12.514196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7e096b9-8d20-49c2-b4ca-22041fb503fc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.514196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514184 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5e16cd1-ac7f-419f-a206-ee417e748911-tmp\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.514364 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.514208 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a5e16cd1-ac7f-419f-a206-ee417e748911-snapshots\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.525365 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.525340 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-ggnwn"] Jul 14 09:27:12.525511 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.525495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:12.528718 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.528700 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 14 09:27:12.528845 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.528761 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 14 09:27:12.528910 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.528900 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-q5qvm\"" Jul 14 09:27:12.549473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.549449 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d68dfd9cb-dkqs8"] Jul 14 09:27:12.549600 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.549562 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.551884 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.551866 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.551969 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.551909 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.552051 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.552033 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kc4jt\"" Jul 14 09:27:12.552138 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.552125 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 14 09:27:12.552199 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.552149 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 14 09:27:12.557178 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.557159 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 14 09:27:12.563546 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.563530 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8"] Jul 14 09:27:12.563622 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.563556 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-x4gsp"] Jul 14 09:27:12.563622 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.563570 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4"] Jul 14 09:27:12.563622 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.563581 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-ggnwn"] Jul 14 09:27:12.563622 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.563593 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d68dfd9cb-dkqs8"] Jul 14 09:27:12.563763 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.563651 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.565329 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.565309 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg"] Jul 14 09:27:12.566114 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566096 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 14 09:27:12.566255 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566242 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 14 09:27:12.566456 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566436 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.566529 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566484 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.566593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566575 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 14 09:27:12.566713 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566695 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 14 09:27:12.567004 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.566988 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wtk75\"" Jul 14 09:27:12.581401 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.581372 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw"] Jul 14 09:27:12.581548 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.581529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.583769 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.583733 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 09:27:12.584427 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.584410 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.584427 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.584422 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-d598s\"" Jul 14 09:27:12.584568 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.584428 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 09:27:12.584568 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.584447 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.607001 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.606977 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg"] Jul 14 09:27:12.607001 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.607007 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw"] Jul 14 09:27:12.607184 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.607104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:12.609318 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.609297 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qfhgr\"" Jul 14 09:27:12.609421 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.609297 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:27:12.609476 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.609296 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 09:27:12.609531 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.609298 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 09:27:12.614583 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf2zc\" (UniqueName: \"kubernetes.io/projected/65e9caf5-77ad-46a6-bd56-3f062570093a-kube-api-access-jf2zc\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.614583 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cclv7\" (UniqueName: \"kubernetes.io/projected/a5e16cd1-ac7f-419f-a206-ee417e748911-kube-api-access-cclv7\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.614716 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz65b\" (UniqueName: \"kubernetes.io/projected/ba77dcda-507d-47ee-94ed-e57334e8e4b7-kube-api-access-gz65b\") pod \"network-check-source-869dcfcdcd-2jw44\" (UID: \"ba77dcda-507d-47ee-94ed-e57334e8e4b7\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" Jul 14 09:27:12.614716 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a5e16cd1-ac7f-419f-a206-ee417e748911-snapshots\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.614716 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65e9caf5-77ad-46a6-bd56-3f062570093a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.614885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614757 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:12.614885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5e16cd1-ac7f-419f-a206-ee417e748911-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.614885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08b281cb-bb74-42c1-94a0-4e8f6e97af98-nginx-conf\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:12.614885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614860 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.615050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5e16cd1-ac7f-419f-a206-ee417e748911-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.615050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfsw8\" (UniqueName: \"kubernetes.io/projected/c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db-kube-api-access-pfsw8\") pod \"volume-data-source-validator-78495597f6-skbx8\" (UID: \"c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" Jul 14 09:27:12.615050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5e16cd1-ac7f-419f-a206-ee417e748911-serving-cert\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.615050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614971 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7e096b9-8d20-49c2-b4ca-22041fb503fc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.615050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.614998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxk46\" (UniqueName: \"kubernetes.io/projected/a7e096b9-8d20-49c2-b4ca-22041fb503fc-kube-api-access-xxk46\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.615050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7e096b9-8d20-49c2-b4ca-22041fb503fc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.615331 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615063 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5e16cd1-ac7f-419f-a206-ee417e748911-tmp\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.615331 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615268 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a5e16cd1-ac7f-419f-a206-ee417e748911-snapshots\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.615551 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615529 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7e096b9-8d20-49c2-b4ca-22041fb503fc-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.615599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615542 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5e16cd1-ac7f-419f-a206-ee417e748911-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.615838 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615814 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5e16cd1-ac7f-419f-a206-ee417e748911-tmp\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.615946 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.615814 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5e16cd1-ac7f-419f-a206-ee417e748911-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.617602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.617581 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7e096b9-8d20-49c2-b4ca-22041fb503fc-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.617602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.617599 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5e16cd1-ac7f-419f-a206-ee417e748911-serving-cert\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.624393 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.624362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxk46\" (UniqueName: \"kubernetes.io/projected/a7e096b9-8d20-49c2-b4ca-22041fb503fc-kube-api-access-xxk46\") pod \"kube-storage-version-migrator-operator-799cb7f958-sxzmc\" (UID: \"a7e096b9-8d20-49c2-b4ca-22041fb503fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.625507 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.625488 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz65b\" (UniqueName: \"kubernetes.io/projected/ba77dcda-507d-47ee-94ed-e57334e8e4b7-kube-api-access-gz65b\") pod \"network-check-source-869dcfcdcd-2jw44\" (UID: \"ba77dcda-507d-47ee-94ed-e57334e8e4b7\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" Jul 14 09:27:12.625705 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.625688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cclv7\" (UniqueName: \"kubernetes.io/projected/a5e16cd1-ac7f-419f-a206-ee417e748911-kube-api-access-cclv7\") pod \"insights-operator-68d9dd94cf-tkhdz\" (UID: \"a5e16cd1-ac7f-419f-a206-ee417e748911\") " pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.641624 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.641598 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:12.693580 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.693550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" Jul 14 09:27:12.713732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.713705 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" Jul 14 09:27:12.716006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.715983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.716114 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/277ce53c-c5bb-4e1b-841f-cce8926938a4-config\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.716114 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfsw8\" (UniqueName: \"kubernetes.io/projected/c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db-kube-api-access-pfsw8\") pod \"volume-data-source-validator-78495597f6-skbx8\" (UID: \"c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" Jul 14 09:27:12.716214 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.716122 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:12.716214 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-default-certificate\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.716214 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716155 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:12.716214 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/277ce53c-c5bb-4e1b-841f-cce8926938a4-trusted-ca\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.716357 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.716263 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls podName:65e9caf5-77ad-46a6-bd56-3f062570093a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:13.216243427 +0000 UTC m=+44.154093880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-j6dt4" (UID: "65e9caf5-77ad-46a6-bd56-3f062570093a") : secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:12.716357 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf2zc\" (UniqueName: \"kubernetes.io/projected/65e9caf5-77ad-46a6-bd56-3f062570093a-kube-api-access-jf2zc\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.716448 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swfjk\" (UniqueName: \"kubernetes.io/projected/76baa4da-b647-4d01-81b8-9ce796611b65-kube-api-access-swfjk\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.716448 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76baa4da-b647-4d01-81b8-9ce796611b65-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.716544 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.716544 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716499 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rqht\" (UniqueName: \"kubernetes.io/projected/8e2cf258-7b2c-492f-926c-726447c980f4-kube-api-access-9rqht\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:12.716544 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8zmk\" (UniqueName: \"kubernetes.io/projected/839ee682-38ac-41bd-9798-cef813d75122-kube-api-access-b8zmk\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.716659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716557 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-stats-auth\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.716659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65e9caf5-77ad-46a6-bd56-3f062570093a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.716659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxp2v\" (UniqueName: \"kubernetes.io/projected/277ce53c-c5bb-4e1b-841f-cce8926938a4-kube-api-access-zxp2v\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.716771 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/76baa4da-b647-4d01-81b8-9ce796611b65-config\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.716812 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/277ce53c-c5bb-4e1b-841f-cce8926938a4-serving-cert\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.716812 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:12.716871 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716818 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.716871 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.716838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08b281cb-bb74-42c1-94a0-4e8f6e97af98-nginx-conf\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:12.716933 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.716893 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 09:27:12.716965 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.716944 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert podName:08b281cb-bb74-42c1-94a0-4e8f6e97af98 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:13.216930156 +0000 UTC m=+44.154780607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-x4gsp" (UID: "08b281cb-bb74-42c1-94a0-4e8f6e97af98") : secret "networking-console-plugin-cert" not found Jul 14 09:27:12.717396 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.717378 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08b281cb-bb74-42c1-94a0-4e8f6e97af98-nginx-conf\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:12.717435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.717404 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65e9caf5-77ad-46a6-bd56-3f062570093a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.731166 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.731135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf2zc\" (UniqueName: \"kubernetes.io/projected/65e9caf5-77ad-46a6-bd56-3f062570093a-kube-api-access-jf2zc\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:12.731166 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.731156 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfsw8\" (UniqueName: \"kubernetes.io/projected/c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db-kube-api-access-pfsw8\") pod \"volume-data-source-validator-78495597f6-skbx8\" (UID: \"c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" Jul 14 09:27:12.738951 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.738935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" Jul 14 09:27:12.752664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.752639 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:12.790240 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.790199 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" Jul 14 09:27:12.812317 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.812285 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818606 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76baa4da-b647-4d01-81b8-9ce796611b65-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rqht\" (UniqueName: \"kubernetes.io/projected/8e2cf258-7b2c-492f-926c-726447c980f4-kube-api-access-9rqht\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818721 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8zmk\" (UniqueName: \"kubernetes.io/projected/839ee682-38ac-41bd-9798-cef813d75122-kube-api-access-b8zmk\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-stats-auth\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxp2v\" (UniqueName: \"kubernetes.io/projected/277ce53c-c5bb-4e1b-841f-cce8926938a4-kube-api-access-zxp2v\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/76baa4da-b647-4d01-81b8-9ce796611b65-config\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/277ce53c-c5bb-4e1b-841f-cce8926938a4-serving-cert\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.818990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/277ce53c-c5bb-4e1b-841f-cce8926938a4-config\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.819044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.819042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-default-certificate\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.819739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.819072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:12.819739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.819100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/277ce53c-c5bb-4e1b-841f-cce8926938a4-trusted-ca\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.819739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.819174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swfjk\" (UniqueName: \"kubernetes.io/projected/76baa4da-b647-4d01-81b8-9ce796611b65-kube-api-access-swfjk\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.820696 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.820563 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/76baa4da-b647-4d01-81b8-9ce796611b65-config\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.820696 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.820673 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 09:27:12.820876 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.820731 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:13.320711747 +0000 UTC m=+44.258562200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : secret "router-metrics-certs-default" not found Jul 14 09:27:12.823684 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.822605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76baa4da-b647-4d01-81b8-9ce796611b65-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.823684 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.822655 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:13.32263778 +0000 UTC m=+44.260488332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : configmap references non-existent config key: service-ca.crt Jul 14 09:27:12.823684 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.822712 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 09:27:12.823684 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:12.822782 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls podName:8e2cf258-7b2c-492f-926c-726447c980f4 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:13.322765736 +0000 UTC m=+44.260616198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls") pod "cluster-samples-operator-65db858894-hfxlw" (UID: "8e2cf258-7b2c-492f-926c-726447c980f4") : secret "samples-operator-tls" not found Jul 14 09:27:12.823684 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.823377 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/277ce53c-c5bb-4e1b-841f-cce8926938a4-config\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.823684 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.823595 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/277ce53c-c5bb-4e1b-841f-cce8926938a4-trusted-ca\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.825182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.825094 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/277ce53c-c5bb-4e1b-841f-cce8926938a4-serving-cert\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.826311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.826229 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-stats-auth\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.828524 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.828502 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-default-certificate\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.829294 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.829249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swfjk\" (UniqueName: \"kubernetes.io/projected/76baa4da-b647-4d01-81b8-9ce796611b65-kube-api-access-swfjk\") pod \"service-ca-operator-7695fc6c8c-hpqqg\" (UID: \"76baa4da-b647-4d01-81b8-9ce796611b65\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.830888 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.830772 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8zmk\" (UniqueName: \"kubernetes.io/projected/839ee682-38ac-41bd-9798-cef813d75122-kube-api-access-b8zmk\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:12.830888 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.830807 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rqht\" (UniqueName: \"kubernetes.io/projected/8e2cf258-7b2c-492f-926c-726447c980f4-kube-api-access-9rqht\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:12.832319 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.832257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxp2v\" (UniqueName: \"kubernetes.io/projected/277ce53c-c5bb-4e1b-841f-cce8926938a4-kube-api-access-zxp2v\") pod \"console-operator-7866497494-ggnwn\" (UID: \"277ce53c-c5bb-4e1b-841f-cce8926938a4\") " pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.859436 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.859403 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:12.866451 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.866367 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44"] Jul 14 09:27:12.871672 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.871472 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:12.872104 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:12.872063 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba77dcda_507d_47ee_94ed_e57334e8e4b7.slice/crio-0b6ce3a986e38f6cf91b6f27fe762649b41557bd5f37110ad89a66c452ad1ca2 WatchSource:0}: Error finding container 0b6ce3a986e38f6cf91b6f27fe762649b41557bd5f37110ad89a66c452ad1ca2: Status 404 returned error can't find the container with id 0b6ce3a986e38f6cf91b6f27fe762649b41557bd5f37110ad89a66c452ad1ca2 Jul 14 09:27:12.893162 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.893135 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" Jul 14 09:27:12.907231 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.906931 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc"] Jul 14 09:27:12.945907 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.945821 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" event={"ID":"ba77dcda-507d-47ee-94ed-e57334e8e4b7","Type":"ContainerStarted","Data":"0b6ce3a986e38f6cf91b6f27fe762649b41557bd5f37110ad89a66c452ad1ca2"} Jul 14 09:27:12.948177 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.948114 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" event={"ID":"a7e096b9-8d20-49c2-b4ca-22041fb503fc","Type":"ContainerStarted","Data":"14dc3042bf21576202bfe25eff3f8a1d9e9fe6997dc547e315617f72e8bf2522"} Jul 14 09:27:12.959866 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.959819 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tkhdz"] Jul 14 09:27:12.982638 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:12.982602 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8"] Jul 14 09:27:12.987850 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:12.987637 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37a3ab6_cc1c_45cc_a9a1_9bd2d50d51db.slice/crio-c3640f138487becf5e6396b6f2f37884da6e52baa9211686c9639e787d2098c8 WatchSource:0}: Error finding container c3640f138487becf5e6396b6f2f37884da6e52baa9211686c9639e787d2098c8: Status 404 returned error can't find the container with id c3640f138487becf5e6396b6f2f37884da6e52baa9211686c9639e787d2098c8 Jul 14 09:27:13.048316 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.048292 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-ggnwn"] Jul 14 09:27:13.050673 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:13.050644 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277ce53c_c5bb_4e1b_841f_cce8926938a4.slice/crio-64a86f692201a6cc73e32999f7a3116a49c4eeebc0d0ad6a255f76034b18351e WatchSource:0}: Error finding container 64a86f692201a6cc73e32999f7a3116a49c4eeebc0d0ad6a255f76034b18351e: Status 404 returned error can't find the container with id 64a86f692201a6cc73e32999f7a3116a49c4eeebc0d0ad6a255f76034b18351e Jul 14 09:27:13.061834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.061810 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg"] Jul 14 09:27:13.064985 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:13.064962 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76baa4da_b647_4d01_81b8_9ce796611b65.slice/crio-202037ed308d291bfe9de73dec42b8604cb55439796299d34287322361b2bc20 WatchSource:0}: Error finding container 202037ed308d291bfe9de73dec42b8604cb55439796299d34287322361b2bc20: Status 404 returned error can't find the container with id 202037ed308d291bfe9de73dec42b8604cb55439796299d34287322361b2bc20 Jul 14 09:27:13.223188 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.223099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:13.223188 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.223162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:13.223361 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.223255 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 09:27:13.223361 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.223330 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:13.223361 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.223334 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert podName:08b281cb-bb74-42c1-94a0-4e8f6e97af98 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:14.223302874 +0000 UTC m=+45.161153327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-x4gsp" (UID: "08b281cb-bb74-42c1-94a0-4e8f6e97af98") : secret "networking-console-plugin-cert" not found Jul 14 09:27:13.223473 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.223378 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls podName:65e9caf5-77ad-46a6-bd56-3f062570093a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:14.223362749 +0000 UTC m=+45.161213207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-j6dt4" (UID: "65e9caf5-77ad-46a6-bd56-3f062570093a") : secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:13.324077 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.324043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:13.324217 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.324109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:13.324217 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.324191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:13.324315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.324229 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 09:27:13.324315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.324234 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:14.324217141 +0000 UTC m=+45.262067589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : configmap references non-existent config key: service-ca.crt Jul 14 09:27:13.324315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.324272 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 09:27:13.324315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.324297 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:14.324282067 +0000 UTC m=+45.262132514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : secret "router-metrics-certs-default" not found Jul 14 09:27:13.324315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:13.324315 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls podName:8e2cf258-7b2c-492f-926c-726447c980f4 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:14.324305741 +0000 UTC m=+45.262156190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls") pod "cluster-samples-operator-65db858894-hfxlw" (UID: "8e2cf258-7b2c-492f-926c-726447c980f4") : secret "samples-operator-tls" not found Jul 14 09:27:13.958865 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.957835 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" event={"ID":"ba77dcda-507d-47ee-94ed-e57334e8e4b7","Type":"ContainerStarted","Data":"c6ffb64f7dd019173c3d2d11e337881809b0e3d69eeb71c8b6d9d336b76b35e5"} Jul 14 09:27:13.962506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.962474 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" event={"ID":"a5e16cd1-ac7f-419f-a206-ee417e748911","Type":"ContainerStarted","Data":"599ec643ddbf1dee2e36149bda7c7fb0d20ae915594d389b8e38102465d67471"} Jul 14 09:27:13.963950 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.963880 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" event={"ID":"76baa4da-b647-4d01-81b8-9ce796611b65","Type":"ContainerStarted","Data":"202037ed308d291bfe9de73dec42b8604cb55439796299d34287322361b2bc20"} Jul 14 09:27:13.965904 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.965882 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" event={"ID":"c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db","Type":"ContainerStarted","Data":"c3640f138487becf5e6396b6f2f37884da6e52baa9211686c9639e787d2098c8"} Jul 14 09:27:13.968402 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.968373 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ggnwn" event={"ID":"277ce53c-c5bb-4e1b-841f-cce8926938a4","Type":"ContainerStarted","Data":"64a86f692201a6cc73e32999f7a3116a49c4eeebc0d0ad6a255f76034b18351e"} Jul 14 09:27:13.974649 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:13.974099 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2jw44" podStartSLOduration=1.974082324 podStartE2EDuration="1.974082324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:13.972602823 +0000 UTC m=+44.910453296" watchObservedRunningTime="2026-07-14 09:27:13.974082324 +0000 UTC m=+44.911932796" Jul 14 09:27:14.234926 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.234838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:14.235093 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.235015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:14.235199 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.235180 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 09:27:14.235268 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.235256 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert podName:08b281cb-bb74-42c1-94a0-4e8f6e97af98 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:16.235234497 +0000 UTC m=+47.173084945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-x4gsp" (UID: "08b281cb-bb74-42c1-94a0-4e8f6e97af98") : secret "networking-console-plugin-cert" not found Jul 14 09:27:14.235379 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.235336 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:14.235451 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.235379 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls podName:65e9caf5-77ad-46a6-bd56-3f062570093a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:16.235366818 +0000 UTC m=+47.173217277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-j6dt4" (UID: "65e9caf5-77ad-46a6-bd56-3f062570093a") : secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:14.257605 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.257551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:14.336173 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.336135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:14.336346 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.336235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:14.336346 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.336313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:14.336457 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.336443 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 09:27:14.336833 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.336509 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:16.33648942 +0000 UTC m=+47.274339918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : secret "router-metrics-certs-default" not found Jul 14 09:27:14.336833 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.336553 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:16.336531814 +0000 UTC m=+47.274382277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : configmap references non-existent config key: service-ca.crt Jul 14 09:27:14.336833 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.336682 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 09:27:14.336833 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:14.336723 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls podName:8e2cf258-7b2c-492f-926c-726447c980f4 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:16.336711358 +0000 UTC m=+47.274561819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls") pod "cluster-samples-operator-65db858894-hfxlw" (UID: "8e2cf258-7b2c-492f-926c-726447c980f4") : secret "samples-operator-tls" not found Jul 14 09:27:14.357127 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.357073 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:14.457771 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:14.457700 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:15.287182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:15.287131 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:15.388760 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:15.388705 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:16.263367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:16.263324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:16.263597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:16.263392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:16.263597 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.263490 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 09:27:16.263597 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.263554 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert podName:08b281cb-bb74-42c1-94a0-4e8f6e97af98 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.263538232 +0000 UTC m=+51.201388683 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-x4gsp" (UID: "08b281cb-bb74-42c1-94a0-4e8f6e97af98") : secret "networking-console-plugin-cert" not found Jul 14 09:27:16.263597 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.263490 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:16.263834 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.263665 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls podName:65e9caf5-77ad-46a6-bd56-3f062570093a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.263639286 +0000 UTC m=+51.201489747 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-j6dt4" (UID: "65e9caf5-77ad-46a6-bd56-3f062570093a") : secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:16.364888 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:16.364850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:16.364920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:16.364992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.365051 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.365024821 +0000 UTC m=+51.302875283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : configmap references non-existent config key: service-ca.crt Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.365078 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.365078 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.365124 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.365111771 +0000 UTC m=+51.302962223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : secret "router-metrics-certs-default" not found Jul 14 09:27:16.365346 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:16.365172 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls podName:8e2cf258-7b2c-492f-926c-726447c980f4 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.365156671 +0000 UTC m=+51.303007120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls") pod "cluster-samples-operator-65db858894-hfxlw" (UID: "8e2cf258-7b2c-492f-926c-726447c980f4") : secret "samples-operator-tls" not found Jul 14 09:27:17.273249 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:17.273210 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:17.273464 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:17.273261 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:17.273464 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:17.273355 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:17.273464 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:17.273421 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls podName:ce209c62-cc38-41b6-8021-d09ba761e072 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:33.273405629 +0000 UTC m=+64.211256080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls") pod "dns-default-5vbvj" (UID: "ce209c62-cc38-41b6-8021-d09ba761e072") : secret "dns-default-metrics-tls" not found Jul 14 09:27:17.273464 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:17.273365 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:17.273464 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:17.273441 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fc449dc4-ft7gq: secret "image-registry-tls" not found Jul 14 09:27:17.273627 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:17.273470 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls podName:be49dc80-58a0-48ae-851d-42ac05324d71 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:33.273460112 +0000 UTC m=+64.211310560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls") pod "image-registry-76fc449dc4-ft7gq" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71") : secret "image-registry-tls" not found Jul 14 09:27:18.022810 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.022763 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:18.088504 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.088461 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:18.988186 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.987692 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" event={"ID":"76baa4da-b647-4d01-81b8-9ce796611b65","Type":"ContainerStarted","Data":"0545d0b6af706d991723e6d8594275c8a9955461fa3a08d9b6dc077f71afdf07"} Jul 14 09:27:18.989578 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.989550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" event={"ID":"c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db","Type":"ContainerStarted","Data":"9bc739153d79701255ae08fb63f23bc9ea17df55ae6625b1fae976b55e230348"} Jul 14 09:27:18.991323 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.991302 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/0.log" Jul 14 09:27:18.991450 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.991342 2579 generic.go:356] "Generic (PLEG): container finished" podID="277ce53c-c5bb-4e1b-841f-cce8926938a4" containerID="54173de92de7516cdac42f6ed517ffa7559c917e1bf27817af93069d10b107ce" exitCode=255 Jul 14 09:27:18.991450 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.991422 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ggnwn" event={"ID":"277ce53c-c5bb-4e1b-841f-cce8926938a4","Type":"ContainerDied","Data":"54173de92de7516cdac42f6ed517ffa7559c917e1bf27817af93069d10b107ce"} Jul 14 09:27:18.991588 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.991567 2579 scope.go:117] "RemoveContainer" containerID="54173de92de7516cdac42f6ed517ffa7559c917e1bf27817af93069d10b107ce" Jul 14 09:27:18.993569 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.993548 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" event={"ID":"a5e16cd1-ac7f-419f-a206-ee417e748911","Type":"ContainerStarted","Data":"b5bd7b1a77baad1ecba2b540a643ad483dc50b6bac0ca8cbd539dd43faef7a30"} Jul 14 09:27:18.995173 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:18.995141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" event={"ID":"a7e096b9-8d20-49c2-b4ca-22041fb503fc","Type":"ContainerStarted","Data":"e50dc727fa86dcea3cb90670c35328d6e0ee6efc35df9cdbd70f9e3cf4fba9ca"} Jul 14 09:27:19.007776 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:19.005700 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" podStartSLOduration=1.7052785099999999 podStartE2EDuration="7.005684164s" totalImagesPullingTime="5.300405654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:19.002928334 +0000 UTC m=+49.940778817" watchObservedRunningTime="2026-07-14 09:27:19.005684164 +0000 UTC m=+49.943534634" Jul 14 09:27:19.019538 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:19.019300 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" podStartSLOduration=1.571886122 podStartE2EDuration="7.019281128s" totalImagesPullingTime="5.447395006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:19.018407576 +0000 UTC m=+49.956258088" watchObservedRunningTime="2026-07-14 09:27:19.019281128 +0000 UTC m=+49.957131602" Jul 14 09:27:19.034531 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:19.034455 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-skbx8" podStartSLOduration=1.6641876820000001 podStartE2EDuration="7.034428731s" totalImagesPullingTime="5.370241049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:19.033874128 +0000 UTC m=+49.971724600" watchObservedRunningTime="2026-07-14 09:27:19.034428731 +0000 UTC m=+49.972279202" Jul 14 09:27:19.053104 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:19.053053 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" podStartSLOduration=1.667416719 podStartE2EDuration="7.053033653s" totalImagesPullingTime="5.385616934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:19.052202937 +0000 UTC m=+49.990053430" watchObservedRunningTime="2026-07-14 09:27:19.053033653 +0000 UTC m=+49.990884124" Jul 14 09:27:19.115066 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:19.115028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:19.214252 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:19.214219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.001309 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.001275 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:27:20.001795 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.001778 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/0.log" Jul 14 09:27:20.001892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.001817 2579 generic.go:356] "Generic (PLEG): container finished" podID="277ce53c-c5bb-4e1b-841f-cce8926938a4" containerID="c66ab53f71060f652782620888a84e62fb7cebb97632762038c3a7b99d474573" exitCode=255 Jul 14 09:27:20.001892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.001854 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ggnwn" event={"ID":"277ce53c-c5bb-4e1b-841f-cce8926938a4","Type":"ContainerDied","Data":"c66ab53f71060f652782620888a84e62fb7cebb97632762038c3a7b99d474573"} Jul 14 09:27:20.001992 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.001932 2579 scope.go:117] "RemoveContainer" containerID="54173de92de7516cdac42f6ed517ffa7559c917e1bf27817af93069d10b107ce" Jul 14 09:27:20.002174 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.002158 2579 scope.go:117] "RemoveContainer" containerID="c66ab53f71060f652782620888a84e62fb7cebb97632762038c3a7b99d474573" Jul 14 09:27:20.002374 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.002349 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-ggnwn_openshift-console-operator(277ce53c-c5bb-4e1b-841f-cce8926938a4)\"" pod="openshift-console-operator/console-operator-7866497494-ggnwn" podUID="277ce53c-c5bb-4e1b-841f-cce8926938a4" Jul 14 09:27:20.020186 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.020155 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.123549 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.123514 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.247920 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.247882 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nzx54"] Jul 14 09:27:20.252189 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.252138 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.261633 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.261611 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsz5b\"" Jul 14 09:27:20.263343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.263317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.265858 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.265840 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 09:27:20.267431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.267414 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 09:27:20.284302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.284280 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nzx54"] Jul 14 09:27:20.299428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299406 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-crio-socket\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.299571 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-data-volume\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.299571 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.299571 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299562 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:20.299725 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.299725 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:20.299725 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.299668 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 09:27:20.299896 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.299739 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert podName:08b281cb-bb74-42c1-94a0-4e8f6e97af98 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.299721026 +0000 UTC m=+59.237571477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-x4gsp" (UID: "08b281cb-bb74-42c1-94a0-4e8f6e97af98") : secret "networking-console-plugin-cert" not found Jul 14 09:27:20.299896 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.299783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qblcw\" (UniqueName: \"kubernetes.io/projected/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-api-access-qblcw\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.299896 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.299791 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:20.303443 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.300237 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls podName:65e9caf5-77ad-46a6-bd56-3f062570093a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.299842972 +0000 UTC m=+59.237693423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-j6dt4" (UID: "65e9caf5-77ad-46a6-bd56-3f062570093a") : secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:20.371378 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.371341 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.400648 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.400609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.400839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.400669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:20.400839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.400691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.400839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.400763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:20.400839 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400778 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400841 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.400782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qblcw\" (UniqueName: \"kubernetes.io/projected/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-api-access-qblcw\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400864 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.900844784 +0000 UTC m=+51.838695251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400884 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400913 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.400901869 +0000 UTC m=+59.338752321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : configmap references non-existent config key: service-ca.crt Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400930 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:20.900921662 +0000 UTC m=+51.838772122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : configmap "kube-rbac-proxy" not found Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.400976 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-crio-socket\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.401012 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.400997 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls podName:8e2cf258-7b2c-492f-926c-726447c980f4 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.400991002 +0000 UTC m=+59.338841450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls") pod "cluster-samples-operator-65db858894-hfxlw" (UID: "8e2cf258-7b2c-492f-926c-726447c980f4") : secret "samples-operator-tls" not found Jul 14 09:27:20.401264 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.401022 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-data-volume\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.401264 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.401036 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-crio-socket\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.401264 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.401068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:20.401264 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.401160 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 09:27:20.401264 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.401203 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.401191893 +0000 UTC m=+59.339042341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : secret "router-metrics-certs-default" not found Jul 14 09:27:20.401426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.401360 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-data-volume\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.409557 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.409537 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qblcw\" (UniqueName: \"kubernetes.io/projected/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-api-access-qblcw\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.439832 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.439797 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.581459 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.581424 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.681584 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.681550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:20.906156 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.906072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.906315 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:20.906185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:20.906315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.906198 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:20.906315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.906256 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:21.906241448 +0000 UTC m=+52.844091895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : configmap "kube-rbac-proxy" not found Jul 14 09:27:20.906315 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.906304 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:20.906450 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:20.906357 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:21.906345284 +0000 UTC m=+52.844195732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:21.007330 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:21.007302 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:27:21.007632 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:21.007618 2579 scope.go:117] "RemoveContainer" containerID="c66ab53f71060f652782620888a84e62fb7cebb97632762038c3a7b99d474573" Jul 14 09:27:21.007799 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:21.007781 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-ggnwn_openshift-console-operator(277ce53c-c5bb-4e1b-841f-cce8926938a4)\"" pod="openshift-console-operator/console-operator-7866497494-ggnwn" podUID="277ce53c-c5bb-4e1b-841f-cce8926938a4" Jul 14 09:27:21.044911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:21.044877 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:21.144962 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:21.144928 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:21.914140 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:21.914105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:21.914342 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:21.914174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:21.914342 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:21.914263 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:21.914342 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:21.914276 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:21.914342 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:21.914307 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:23.914293674 +0000 UTC m=+54.852144122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : configmap "kube-rbac-proxy" not found Jul 14 09:27:21.914342 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:21.914340 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:23.914322917 +0000 UTC m=+54.852173379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:22.677126 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:22.677093 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:22.859862 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:22.859815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:22.860017 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:22.859891 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:22.860233 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:22.860221 2579 scope.go:117] "RemoveContainer" containerID="c66ab53f71060f652782620888a84e62fb7cebb97632762038c3a7b99d474573" Jul 14 09:27:22.860387 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:22.860369 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-ggnwn_openshift-console-operator(277ce53c-c5bb-4e1b-841f-cce8926938a4)\"" pod="openshift-console-operator/console-operator-7866497494-ggnwn" podUID="277ce53c-c5bb-4e1b-841f-cce8926938a4" Jul 14 09:27:22.880030 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:22.880007 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zxfnl_59d7060b-7f01-4a2f-9e55-503176c06e81/dns-node-resolver/0.log" Jul 14 09:27:23.557115 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:23.557081 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:23.881060 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:23.880986 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cglm5_fa319f90-fab6-446b-b19f-cf24a7b53761/node-ca/0.log" Jul 14 09:27:23.932029 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:23.931995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:23.932159 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:23.932071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:23.932218 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:23.932158 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:23.932218 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:23.932190 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:23.932284 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:23.932229 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:27.932208325 +0000 UTC m=+58.870058789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:23.932284 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:23.932245 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:27.932238431 +0000 UTC m=+58.870088878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : configmap "kube-rbac-proxy" not found Jul 14 09:27:24.281119 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:24.281086 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j9jpq_259e96dd-ba4d-4586-8680-ffc219915b18/serve-healthcheck-canary/0.log" Jul 14 09:27:24.339550 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:24.339516 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:25.083142 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:25.083113 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-sxzmc_a7e096b9-8d20-49c2-b4ca-22041fb503fc/kube-storage-version-migrator-operator/0.log" Jul 14 09:27:27.340128 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.340094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:27.444171 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.444130 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:27.595018 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.594935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:27.694106 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.694076 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:27.900584 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.900512 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4bjh8" Jul 14 09:27:27.906319 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.906284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:27.969949 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.969910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:27.970140 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:27.969996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:27.970140 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:27.970115 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:27.970267 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:27.970193 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:35.970171503 +0000 UTC m=+66.908021958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:27.970267 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:27.970117 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:27.970378 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:27.970293 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:35.970273813 +0000 UTC m=+66.908124262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : configmap "kube-rbac-proxy" not found Jul 14 09:27:28.374710 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.374678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:28.375089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.374734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:28.375089 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.374852 2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 09:27:28.375089 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.374866 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:28.375089 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.374925 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert podName:08b281cb-bb74-42c1-94a0-4e8f6e97af98 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:44.37490721 +0000 UTC m=+75.312757661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-x4gsp" (UID: "08b281cb-bb74-42c1-94a0-4e8f6e97af98") : secret "networking-console-plugin-cert" not found Jul 14 09:27:28.375089 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.374944 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls podName:65e9caf5-77ad-46a6-bd56-3f062570093a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:44.374935508 +0000 UTC m=+75.312785957 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-j6dt4" (UID: "65e9caf5-77ad-46a6-bd56-3f062570093a") : secret "cluster-monitoring-operator-tls" not found Jul 14 09:27:28.475182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.475144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:28.475373 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.475289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:28.475373 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.475303 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 09:27:28.475373 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.475338 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:28.475373 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.475373 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls podName:8e2cf258-7b2c-492f-926c-726447c980f4 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:44.475353066 +0000 UTC m=+75.413203531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls") pod "cluster-samples-operator-65db858894-hfxlw" (UID: "8e2cf258-7b2c-492f-926c-726447c980f4") : secret "samples-operator-tls" not found Jul 14 09:27:28.475552 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.475415 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 09:27:28.475552 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.475428 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:44.475416279 +0000 UTC m=+75.413266726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : configmap references non-existent config key: service-ca.crt Jul 14 09:27:28.475552 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:28.475458 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs podName:839ee682-38ac-41bd-9798-cef813d75122 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:44.475445302 +0000 UTC m=+75.413295749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs") pod "router-default-6d68dfd9cb-dkqs8" (UID: "839ee682-38ac-41bd-9798-cef813d75122") : secret "router-metrics-certs-default" not found Jul 14 09:27:28.542283 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.542245 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:28.943708 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:28.943669 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:29.344642 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:29.344597 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:29.744203 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:29.744118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:29.944660 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:29.944631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:30.343432 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:30.343395 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.051658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.051621 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.153473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.153445 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.220499 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.220467 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.308767 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.308665 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.551151 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.551118 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.742141 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.742061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:31.948152 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:31.948115 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:32.006068 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:32.005989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:32.143238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:32.143204 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:33.149505 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.149471 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:33.316294 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.316258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:33.316463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.316318 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:33.318622 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.318596 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce209c62-cc38-41b6-8021-d09ba761e072-metrics-tls\") pod \"dns-default-5vbvj\" (UID: \"ce209c62-cc38-41b6-8021-d09ba761e072\") " pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:33.318764 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.318735 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"image-registry-76fc449dc4-ft7gq\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:33.387368 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.387332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:33.540717 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.540691 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-prphk\"" Jul 14 09:27:33.545778 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.545725 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:33.549309 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.549291 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:33.575364 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.575346 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jncc8\"" Jul 14 09:27:33.583408 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.583390 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:33.633362 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.633126 2579 scope.go:117] "RemoveContainer" containerID="c66ab53f71060f652782620888a84e62fb7cebb97632762038c3a7b99d474573" Jul 14 09:27:33.728347 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.728314 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76fc449dc4-ft7gq"] Jul 14 09:27:33.731337 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:33.731310 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe49dc80_58a0_48ae_851d_42ac05324d71.slice/crio-a3ddbb85cfda33340dbcde892f44b9a8e19393aeb90a627cc488eebf476b108e WatchSource:0}: Error finding container a3ddbb85cfda33340dbcde892f44b9a8e19393aeb90a627cc488eebf476b108e: Status 404 returned error can't find the container with id a3ddbb85cfda33340dbcde892f44b9a8e19393aeb90a627cc488eebf476b108e Jul 14 09:27:33.740785 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.740734 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:33.744367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.744255 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5vbvj"] Jul 14 09:27:33.751094 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:33.751066 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce209c62_cc38_41b6_8021_d09ba761e072.slice/crio-b8874c5e53fb3a4d9efd42d2713cf939bb4d3e004d3dbd8abe8d509355f2f256 WatchSource:0}: Error finding container b8874c5e53fb3a4d9efd42d2713cf939bb4d3e004d3dbd8abe8d509355f2f256: Status 404 returned error can't find the container with id b8874c5e53fb3a4d9efd42d2713cf939bb4d3e004d3dbd8abe8d509355f2f256 Jul 14 09:27:33.838259 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.838220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:33.942628 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:33.942589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:34.049313 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.049276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" event={"ID":"be49dc80-58a0-48ae-851d-42ac05324d71","Type":"ContainerStarted","Data":"bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a"} Jul 14 09:27:34.049516 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.049335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" event={"ID":"be49dc80-58a0-48ae-851d-42ac05324d71","Type":"ContainerStarted","Data":"a3ddbb85cfda33340dbcde892f44b9a8e19393aeb90a627cc488eebf476b108e"} Jul 14 09:27:34.049516 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.049378 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:34.051272 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.051252 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:27:34.051376 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.051357 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-ggnwn" event={"ID":"277ce53c-c5bb-4e1b-841f-cce8926938a4","Type":"ContainerStarted","Data":"0ea3ddab6f0f4d867a83540843e82c35115556abaad13048e2fe6c9c4fe14062"} Jul 14 09:27:34.051693 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.051670 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:34.052584 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.052557 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vbvj" event={"ID":"ce209c62-cc38-41b6-8021-d09ba761e072","Type":"ContainerStarted","Data":"b8874c5e53fb3a4d9efd42d2713cf939bb4d3e004d3dbd8abe8d509355f2f256"} Jul 14 09:27:34.067126 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.067027 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" podStartSLOduration=64.067007631 podStartE2EDuration="1m4.067007631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:34.066701647 +0000 UTC m=+65.004552117" watchObservedRunningTime="2026-07-14 09:27:34.067007631 +0000 UTC m=+65.004858102" Jul 14 09:27:34.082910 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.082862 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-ggnwn" podStartSLOduration=16.77386733 podStartE2EDuration="22.08284449s" totalImagesPullingTime="5.30897716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:34.082424602 +0000 UTC m=+65.020275084" watchObservedRunningTime="2026-07-14 09:27:34.08284449 +0000 UTC m=+65.020694981" Jul 14 09:27:34.146719 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.146686 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:34.328757 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.328667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:27:34.329173 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:34.328793 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 09:27:34.329173 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:34.328865 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs podName:1bcf77f5-947b-4432-b408-425e3e0612a6 nodeName:}" failed. No retries permitted until 2026-07-14 09:28:38.328844575 +0000 UTC m=+129.266695037 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs") pod "network-metrics-daemon-fpqzv" (UID: "1bcf77f5-947b-4432-b408-425e3e0612a6") : secret "metrics-daemon-secret" not found Jul 14 09:27:34.731279 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.731191 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:34.845812 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:34.845780 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-ggnwn" Jul 14 09:27:35.022268 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.022234 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.124215 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.124183 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.223373 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.223335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.323150 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.323056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.503992 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.503935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.601676 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.601603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.704099 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.704065 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.802325 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.802294 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:35.943110 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:35.943036 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.046103 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.046068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:36.046237 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.046171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:36.046298 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:36.046246 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:36.046350 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:36.046325 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls podName:cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa nodeName:}" failed. No retries permitted until 2026-07-14 09:27:52.046302816 +0000 UTC m=+82.984153272 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzx54" (UID: "cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:36.046804 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.046778 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:36.107008 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.106979 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.208968 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.208893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.331880 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.331846 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.386797 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.386738 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.712320 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.712287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.911419 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.911386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:36.976287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:36.976204 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:37.064991 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.064957 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vbvj" event={"ID":"ce209c62-cc38-41b6-8021-d09ba761e072","Type":"ContainerStarted","Data":"4506de65b2f070e44e9df891444616de4c1fcf5ed6761ea5c65db2b3c43da906"} Jul 14 09:27:37.065177 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.065036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vbvj" event={"ID":"ce209c62-cc38-41b6-8021-d09ba761e072","Type":"ContainerStarted","Data":"07f141a694088f239ebe70e8578860ab5751754ba7a58bd5057da9be56f9aaa4"} Jul 14 09:27:37.065177 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.065085 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:37.084615 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.084566 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5vbvj" podStartSLOduration=33.842067649 podStartE2EDuration="36.084551794s" totalImagesPullingTime="2.242484145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:37.084043796 +0000 UTC m=+68.021894267" watchObservedRunningTime="2026-07-14 09:27:37.084551794 +0000 UTC m=+68.022402264" Jul 14 09:27:37.345911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.345880 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:37.543263 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.543230 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:37.948418 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:37.948385 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:38.128725 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:38.128690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:38.345544 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:38.345512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:38.541411 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:38.541362 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:39.596320 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:39.596293 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:39.657289 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:39.657255 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:40.939336 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:40.939305 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8tlqr" Jul 14 09:27:44.419071 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.419001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:44.419071 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.419067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:44.421402 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.421379 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65e9caf5-77ad-46a6-bd56-3f062570093a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-j6dt4\" (UID: \"65e9caf5-77ad-46a6-bd56-3f062570093a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:44.421524 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.421497 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08b281cb-bb74-42c1-94a0-4e8f6e97af98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-x4gsp\" (UID: \"08b281cb-bb74-42c1-94a0-4e8f6e97af98\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:44.505497 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.505459 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-gpwss"] Jul 14 09:27:44.508537 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.508514 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:27:44.510920 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.510901 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-t7xsx\"" Jul 14 09:27:44.511243 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.511225 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 09:27:44.511468 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.511449 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 09:27:44.519868 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.519840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:44.519987 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.519934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:44.520058 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.519995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:44.520886 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.520862 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/839ee682-38ac-41bd-9798-cef813d75122-service-ca-bundle\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:44.522357 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.522337 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gpwss"] Jul 14 09:27:44.522723 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.522703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2cf258-7b2c-492f-926c-726447c980f4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hfxlw\" (UID: \"8e2cf258-7b2c-492f-926c-726447c980f4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:44.522810 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.522703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/839ee682-38ac-41bd-9798-cef813d75122-metrics-certs\") pod \"router-default-6d68dfd9cb-dkqs8\" (UID: \"839ee682-38ac-41bd-9798-cef813d75122\") " pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:44.539495 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.539427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:44.611916 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.611891 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zm2jz\"" Jul 14 09:27:44.620090 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.620067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" Jul 14 09:27:44.620409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.620390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnd5\" (UniqueName: \"kubernetes.io/projected/d722f4e1-d64e-4e48-ba2e-1e8f006e89ee-kube-api-access-fxnd5\") pod \"downloads-68c9cbb497-gpwss\" (UID: \"d722f4e1-d64e-4e48-ba2e-1e8f006e89ee\") " pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:27:44.621269 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.621247 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:44.636003 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.635985 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-q5qvm\"" Jul 14 09:27:44.644330 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.644305 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" Jul 14 09:27:44.674605 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.674580 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wtk75\"" Jul 14 09:27:44.684099 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.684064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:44.718157 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.717781 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qfhgr\"" Jul 14 09:27:44.718785 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.718666 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:44.722397 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.721917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnd5\" (UniqueName: \"kubernetes.io/projected/d722f4e1-d64e-4e48-ba2e-1e8f006e89ee-kube-api-access-fxnd5\") pod \"downloads-68c9cbb497-gpwss\" (UID: \"d722f4e1-d64e-4e48-ba2e-1e8f006e89ee\") " pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:27:44.726139 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.725877 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" Jul 14 09:27:44.737042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.737008 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnd5\" (UniqueName: \"kubernetes.io/projected/d722f4e1-d64e-4e48-ba2e-1e8f006e89ee-kube-api-access-fxnd5\") pod \"downloads-68c9cbb497-gpwss\" (UID: \"d722f4e1-d64e-4e48-ba2e-1e8f006e89ee\") " pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:27:44.768716 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.768529 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4"] Jul 14 09:27:44.772311 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:44.772268 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e9caf5_77ad_46a6_bd56_3f062570093a.slice/crio-ae7f7d3d8a5006d691732814625f2ab39985700047b22a367ef2e644e7b9c074 WatchSource:0}: Error finding container ae7f7d3d8a5006d691732814625f2ab39985700047b22a367ef2e644e7b9c074: Status 404 returned error can't find the container with id ae7f7d3d8a5006d691732814625f2ab39985700047b22a367ef2e644e7b9c074 Jul 14 09:27:44.801588 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.801558 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-x4gsp"] Jul 14 09:27:44.806316 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.806214 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:44.808087 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:44.808058 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b281cb_bb74_42c1_94a0_4e8f6e97af98.slice/crio-1496a3fab718b6db26691981cc2d458795a5736aeb35464dc65186c212db1548 WatchSource:0}: Error finding container 1496a3fab718b6db26691981cc2d458795a5736aeb35464dc65186c212db1548: Status 404 returned error can't find the container with id 1496a3fab718b6db26691981cc2d458795a5736aeb35464dc65186c212db1548 Jul 14 09:27:44.818168 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.818145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:27:44.842960 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.842931 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d68dfd9cb-dkqs8"] Jul 14 09:27:44.845822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:44.845788 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839ee682_38ac_41bd_9798_cef813d75122.slice/crio-f0de99ccb7f1b98f6431afffad7fd294eeea0e4e21b0f09a866a636439587daf WatchSource:0}: Error finding container f0de99ccb7f1b98f6431afffad7fd294eeea0e4e21b0f09a866a636439587daf: Status 404 returned error can't find the container with id f0de99ccb7f1b98f6431afffad7fd294eeea0e4e21b0f09a866a636439587daf Jul 14 09:27:44.891705 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.891657 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw"] Jul 14 09:27:44.966028 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:44.965995 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gpwss"] Jul 14 09:27:44.969225 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:44.969176 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd722f4e1_d64e_4e48_ba2e_1e8f006e89ee.slice/crio-bc82ba49812dbc258fdfbcd54726ba673032b9a22274505ddd4137bbfd96702c WatchSource:0}: Error finding container bc82ba49812dbc258fdfbcd54726ba673032b9a22274505ddd4137bbfd96702c: Status 404 returned error can't find the container with id bc82ba49812dbc258fdfbcd54726ba673032b9a22274505ddd4137bbfd96702c Jul 14 09:27:45.093115 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.093077 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" event={"ID":"65e9caf5-77ad-46a6-bd56-3f062570093a","Type":"ContainerStarted","Data":"ae7f7d3d8a5006d691732814625f2ab39985700047b22a367ef2e644e7b9c074"} Jul 14 09:27:45.094342 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.094311 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" event={"ID":"8e2cf258-7b2c-492f-926c-726447c980f4","Type":"ContainerStarted","Data":"a70231eb7731e2abe6d372a779ce18c983ca1345344518b4fd46e3616960d809"} Jul 14 09:27:45.095811 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.095779 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" event={"ID":"08b281cb-bb74-42c1-94a0-4e8f6e97af98","Type":"ContainerStarted","Data":"1496a3fab718b6db26691981cc2d458795a5736aeb35464dc65186c212db1548"} Jul 14 09:27:45.096895 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.096876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gpwss" event={"ID":"d722f4e1-d64e-4e48-ba2e-1e8f006e89ee","Type":"ContainerStarted","Data":"bc82ba49812dbc258fdfbcd54726ba673032b9a22274505ddd4137bbfd96702c"} Jul 14 09:27:45.098300 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.098281 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" event={"ID":"839ee682-38ac-41bd-9798-cef813d75122","Type":"ContainerStarted","Data":"0196fac1bb05747eb379835550ca7e1361b63e88a16026364d8389863d958e5a"} Jul 14 09:27:45.098366 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.098318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" event={"ID":"839ee682-38ac-41bd-9798-cef813d75122","Type":"ContainerStarted","Data":"f0de99ccb7f1b98f6431afffad7fd294eeea0e4e21b0f09a866a636439587daf"} Jul 14 09:27:45.116879 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.116831 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" podStartSLOduration=33.116813959 podStartE2EDuration="33.116813959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:45.116157774 +0000 UTC m=+76.054008243" watchObservedRunningTime="2026-07-14 09:27:45.116813959 +0000 UTC m=+76.054664430" Jul 14 09:27:45.434822 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.434777 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:45.511921 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.511863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:45.569225 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.569175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:45.685434 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.685334 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:45.688648 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.688441 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:45.859995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.859963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:45.918707 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:45.918674 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:46.104728 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:46.104646 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:46.106489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:46.106292 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d68dfd9cb-dkqs8" Jul 14 09:27:47.070834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:47.070806 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5vbvj" Jul 14 09:27:47.833430 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:47.833355 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:47.933427 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:47.933390 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:47.998965 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:47.998854 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:48.118348 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.117835 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" event={"ID":"65e9caf5-77ad-46a6-bd56-3f062570093a","Type":"ContainerStarted","Data":"c6f58c7545e02460685c43629e91cd814cb1dc35c173f948efd5d4be960774f1"} Jul 14 09:27:48.120061 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.120029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" event={"ID":"8e2cf258-7b2c-492f-926c-726447c980f4","Type":"ContainerStarted","Data":"4abcc971b2e7d15e27e9432bc8964294c875d0d389348991f694da7590162e5d"} Jul 14 09:27:48.120182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.120079 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" event={"ID":"8e2cf258-7b2c-492f-926c-726447c980f4","Type":"ContainerStarted","Data":"2721fdc563ca282d9ebfed9da5ed8cd31e625bf30b9d5eb1bbcca4ea8d110e2b"} Jul 14 09:27:48.122122 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.122096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" event={"ID":"08b281cb-bb74-42c1-94a0-4e8f6e97af98","Type":"ContainerStarted","Data":"bf4a68ed8e85aa77a80bf85f4c964de58805b1106022253288ce1dcaa638f920"} Jul 14 09:27:48.139394 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.139339 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-j6dt4" podStartSLOduration=33.468614109 podStartE2EDuration="36.139319635s" totalImagesPullingTime="2.670705526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:48.138657159 +0000 UTC m=+79.076507629" watchObservedRunningTime="2026-07-14 09:27:48.139319635 +0000 UTC m=+79.077170106" Jul 14 09:27:48.158255 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.158211 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-x4gsp" podStartSLOduration=33.527604365 podStartE2EDuration="36.15819724s" totalImagesPullingTime="2.630592875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:48.156658309 +0000 UTC m=+79.094508779" watchObservedRunningTime="2026-07-14 09:27:48.15819724 +0000 UTC m=+79.096047710" Jul 14 09:27:48.174229 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.174184 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hfxlw" podStartSLOduration=33.670728059 podStartE2EDuration="36.174173005s" totalImagesPullingTime="2.503444946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:48.173459866 +0000 UTC m=+79.111310335" watchObservedRunningTime="2026-07-14 09:27:48.174173005 +0000 UTC m=+79.112023474" Jul 14 09:27:48.302034 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.301985 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:48.375172 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.375089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:48.440926 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:48.440881 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:50.741561 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:50.741528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:50.843076 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:50.842993 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:50.943394 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:50.943354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:51.009388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:51.009352 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:51.090087 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:51.090052 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:51.188917 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:51.188864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:51.287913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:51.287822 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:51.355426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:51.355389 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:52.087333 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.087296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:52.089986 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.089960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzx54\" (UID: \"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa\") " pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:52.330882 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.330847 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:52.363468 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.363390 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsz5b\"" Jul 14 09:27:52.371432 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.371400 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nzx54" Jul 14 09:27:52.384676 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.384648 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:52.446877 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.446576 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:52.520717 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.520685 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nzx54"] Jul 14 09:27:52.524166 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:27:52.524134 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfad1337_ce60_4d45_b5ba_a94f5a9ea0aa.slice/crio-79e7ca2099fb5b2805ebee1f8789086aaf690cca7c7026aa6039a6eca9be4eb3 WatchSource:0}: Error finding container 79e7ca2099fb5b2805ebee1f8789086aaf690cca7c7026aa6039a6eca9be4eb3: Status 404 returned error can't find the container with id 79e7ca2099fb5b2805ebee1f8789086aaf690cca7c7026aa6039a6eca9be4eb3 Jul 14 09:27:52.524485 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:52.524458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:53.147991 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:53.147913 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzx54" event={"ID":"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa","Type":"ContainerStarted","Data":"79e7ca2099fb5b2805ebee1f8789086aaf690cca7c7026aa6039a6eca9be4eb3"} Jul 14 09:27:53.460775 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:53.460651 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:53.556618 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:53.556312 2579 patch_prober.go:28] interesting pod/image-registry-76fc449dc4-ft7gq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 09:27:53.556618 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:53.556371 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" podUID="be49dc80-58a0-48ae-851d-42ac05324d71" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 09:27:53.845764 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:53.845703 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:53.946392 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:53.946334 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:54.058739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:54.058698 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:54.135287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:54.135153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:54.257698 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:54.257639 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:55.062004 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:55.061975 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:27:55.342164 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:55.342078 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:55.442647 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:55.442605 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:55.808409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:55.808372 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:55.891088 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:55.890931 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:56.542391 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:56.542356 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:57.291771 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:57.291712 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.032048 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.032010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.090915 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.090879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.231003 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.230966 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.328466 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.328380 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.444075 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.444039 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.545334 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.545299 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.614636 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.614548 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qzlwt"] Jul 14 09:27:59.619453 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.619428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.622506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.622479 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pkgzk\"" Jul 14 09:27:59.622620 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.622529 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 09:27:59.622692 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.622627 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 09:27:59.622794 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.622776 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 09:27:59.622902 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.622884 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 09:27:59.648278 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.648242 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.658284 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658252 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-accelerators-collector-config\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658395 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658292 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-metrics-client-ca\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658395 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658323 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-root\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658395 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-tls\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658395 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-wtmp\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658647 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658430 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658647 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9nv6\" (UniqueName: \"kubernetes.io/projected/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-kube-api-access-r9nv6\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658647 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-textfile\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.658829 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.658669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-sys\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.749923 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.749859 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.759975 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.759947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-metrics-client-ca\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.759990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-root\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-tls\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-wtmp\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760083 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-root\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760129 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:59.760118 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760140 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9nv6\" (UniqueName: \"kubernetes.io/projected/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-kube-api-access-r9nv6\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:27:59.760174 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-tls podName:2040c5ca-1d7c-4fd5-a660-764faf71cc0c nodeName:}" failed. No retries permitted until 2026-07-14 09:28:00.260158021 +0000 UTC m=+91.198008468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-tls") pod "node-exporter-qzlwt" (UID: "2040c5ca-1d7c-4fd5-a660-764faf71cc0c") : secret "node-exporter-tls" not found Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-textfile\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760213 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-wtmp\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760232 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-sys\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-accelerators-collector-config\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760435 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760371 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-sys\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760826 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760606 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-textfile\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760826 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-metrics-client-ca\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.760826 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.760813 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-accelerators-collector-config\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.763008 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.762972 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.770273 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.770247 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9nv6\" (UniqueName: \"kubernetes.io/projected/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-kube-api-access-r9nv6\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:27:59.848348 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.848291 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:27:59.945721 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:27:59.945602 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:00.045441 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.045404 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:00.146357 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.146319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:00.265454 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.265424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-tls\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:28:00.267884 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.267858 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2040c5ca-1d7c-4fd5-a660-764faf71cc0c-node-exporter-tls\") pod \"node-exporter-qzlwt\" (UID: \"2040c5ca-1d7c-4fd5-a660-764faf71cc0c\") " pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:28:00.486956 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.486917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:00.531777 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.531674 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qzlwt" Jul 14 09:28:00.569354 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.569309 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:00.955360 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:00.955213 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:02.425079 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:02.425041 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:02.556098 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:02.556062 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:02.765416 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:02.765376 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:02.866225 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:02.866190 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:03.440596 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:03.440560 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:03.545489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:03.545459 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:03.571839 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:28:03.571809 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2040c5ca_1d7c_4fd5_a660_764faf71cc0c.slice/crio-545eb9acf4317e5f1255a2ef1b7808dc5ce78c9d767c75a3e63c2394c2b7b5b8 WatchSource:0}: Error finding container 545eb9acf4317e5f1255a2ef1b7808dc5ce78c9d767c75a3e63c2394c2b7b5b8: Status 404 returned error can't find the container with id 545eb9acf4317e5f1255a2ef1b7808dc5ce78c9d767c75a3e63c2394c2b7b5b8 Jul 14 09:28:04.207907 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.207863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gpwss" event={"ID":"d722f4e1-d64e-4e48-ba2e-1e8f006e89ee","Type":"ContainerStarted","Data":"e59285d57d1d8beaefdfee50bbfb0826e9f7cf54bc21c80899474c9b944a187a"} Jul 14 09:28:04.208504 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.208483 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:28:04.212661 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.212629 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzx54" event={"ID":"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa","Type":"ContainerStarted","Data":"829fbe8ca0d0e5ba0056187beb867d239350c52c51a289f4b214064ab28dbd2d"} Jul 14 09:28:04.216423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.216324 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qzlwt" event={"ID":"2040c5ca-1d7c-4fd5-a660-764faf71cc0c","Type":"ContainerStarted","Data":"545eb9acf4317e5f1255a2ef1b7808dc5ce78c9d767c75a3e63c2394c2b7b5b8"} Jul 14 09:28:04.222545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.222517 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-gpwss" Jul 14 09:28:04.277283 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.277239 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:04.304900 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.303642 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-gpwss" podStartSLOduration=1.655875441 podStartE2EDuration="20.303618687s" totalImagesPullingTime="18.647743246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:28:04.230978416 +0000 UTC m=+95.168828889" watchObservedRunningTime="2026-07-14 09:28:04.303618687 +0000 UTC m=+95.241469151" Jul 14 09:28:04.405965 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.405927 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:04.505889 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:04.505694 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:05.039460 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.039416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:05.222930 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.222801 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzx54" event={"ID":"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa","Type":"ContainerStarted","Data":"fa20e18598766479ee063613d53d998d769d574c51455fa39d205d179476f706"} Jul 14 09:28:05.224739 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.224701 2579 generic.go:356] "Generic (PLEG): container finished" podID="2040c5ca-1d7c-4fd5-a660-764faf71cc0c" containerID="58c1501ba9c135fc690040c2d0d2fbf2d407371a9e77f52ad8e88bd27f890b24" exitCode=0 Jul 14 09:28:05.224894 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.224785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qzlwt" event={"ID":"2040c5ca-1d7c-4fd5-a660-764faf71cc0c","Type":"ContainerDied","Data":"58c1501ba9c135fc690040c2d0d2fbf2d407371a9e77f52ad8e88bd27f890b24"} Jul 14 09:28:05.787805 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.787741 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:05.791294 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.791267 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 09:28:05.819697 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.819639 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 09:28:05.820041 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.819824 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.823265 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.823030 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 09:28:05.823265 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.823187 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 09:28:05.823501 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.823484 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 09:28:05.823594 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.823500 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 09:28:05.823681 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.823665 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824024 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824081 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824410 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824583 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824601 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mtffj\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.824938 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.825001 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9t8lacfbqjif7\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.825351 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 09:28:05.826245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.825355 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 09:28:05.827893 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.827860 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 09:28:05.832832 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.832637 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 09:28:05.869341 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.869292 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:05.916710 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916710 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhgqf\" (UniqueName: \"kubernetes.io/projected/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-kube-api-access-fhgqf\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916916 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.916984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.916984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-config\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917085 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917142 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917211 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917247 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917311 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:05.917423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:05.917384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhgqf\" (UniqueName: \"kubernetes.io/projected/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-kube-api-access-fhgqf\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018261 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018306 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018333 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018367 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018447 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-config\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018563 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.018872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018954 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.018980 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.019021 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.019021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.019057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019428 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.019318 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019795 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.019644 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.019795 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.019707 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.020413 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.020383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.023699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.023674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.024326 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.024300 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.027469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.027268 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.027935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.027905 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.029508 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.029465 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhgqf\" (UniqueName: \"kubernetes.io/projected/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-kube-api-access-fhgqf\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.030687 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.030289 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-config\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.030687 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.030493 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.030861 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.030838 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.032595 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.032573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.033801 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.033773 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.033879 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.033795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.034447 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.034421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.035370 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.034980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.035370 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.035327 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.035605 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.035584 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.135796 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.135545 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:06.239030 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.238993 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qzlwt" event={"ID":"2040c5ca-1d7c-4fd5-a660-764faf71cc0c","Type":"ContainerStarted","Data":"19cb7f2d8aeccf15bea2c81ca710d962efe5db0524f500a5405a49032a677eb0"} Jul 14 09:28:06.239200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.239060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qzlwt" event={"ID":"2040c5ca-1d7c-4fd5-a660-764faf71cc0c","Type":"ContainerStarted","Data":"550eab2e01db4993f324f6f94f7b2181c18efa6795fc573608d1d47baba10c31"} Jul 14 09:28:06.333541 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.332397 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qzlwt" podStartSLOduration=6.032845753 podStartE2EDuration="7.332375039s" totalImagesPullingTime="1.22111392s" totalInitContainerRuntime="78.415366ms" isStatefulPod=true podCreationTimestamp="2026-07-14 09:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:28:06.263373492 +0000 UTC m=+97.201223964" watchObservedRunningTime="2026-07-14 09:28:06.332375039 +0000 UTC m=+97.270225510" Jul 14 09:28:06.333541 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:06.333458 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 09:28:06.338029 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:28:06.337921 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10a3b2c_8024_4ed2_9b7f_7a9d5c50bda8.slice/crio-86714fdabf134313776f6ed332b58b534d922222e27ea389a67d863622853157 WatchSource:0}: Error finding container 86714fdabf134313776f6ed332b58b534d922222e27ea389a67d863622853157: Status 404 returned error can't find the container with id 86714fdabf134313776f6ed332b58b534d922222e27ea389a67d863622853157 Jul 14 09:28:07.244825 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:07.244684 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"86714fdabf134313776f6ed332b58b534d922222e27ea389a67d863622853157"} Jul 14 09:28:07.769931 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:07.769887 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76fc449dc4-ft7gq"] Jul 14 09:28:07.894328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:07.894285 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:07.995668 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:07.995627 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:09.255860 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:09.255820 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzx54" event={"ID":"cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa","Type":"ContainerStarted","Data":"b2e20317ea9a569d5034250d64648d3513a0cde9c27b8f89d488a8108c8300c3"} Jul 14 09:28:09.274510 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:09.274442 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nzx54" podStartSLOduration=33.548275003 podStartE2EDuration="49.2744189s" totalImagesPullingTime="15.726143897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:27:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:28:09.273408849 +0000 UTC m=+100.211259330" watchObservedRunningTime="2026-07-14 09:28:09.2744189 +0000 UTC m=+100.212269371" Jul 14 09:28:09.594201 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:09.594167 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:09.694906 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:09.694835 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:10.261522 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:10.261481 2579 generic.go:356] "Generic (PLEG): container finished" podID="d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8" containerID="237b1e9eec19d87ae58e78feb28a1995c2f9fd2f82a29ea97b24ac7378753dab" exitCode=0 Jul 14 09:28:10.262001 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:10.261579 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerDied","Data":"237b1e9eec19d87ae58e78feb28a1995c2f9fd2f82a29ea97b24ac7378753dab"} Jul 14 09:28:10.758013 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:10.757983 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:10.840835 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:10.840798 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:11.373706 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:11.373656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:11.426143 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:11.426065 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:13.807502 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:13.807452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:15.286203 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:15.286168 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"557b6f57138d4ac42446bd3a7d415a7e5e8f837442b0a824f7567dc4161cd4fe"} Jul 14 09:28:15.286613 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:15.286271 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"752fd776e2f1a53976f1cfb17f84d1d49e0cd53e97898836052cb54391a3c2ce"} Jul 14 09:28:16.119146 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:16.119114 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:16.178358 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:16.178320 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:17.296696 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:17.296670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"1bb42b60b84da23c1d1537592bab4e44f6a98370ee5c38494b29ee6b15065ba1"} Jul 14 09:28:18.304444 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:18.304411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"15e0b9d1a92fb695282a56025d8cecdcffab6b84b5d02afc807e8e5a120eb70f"} Jul 14 09:28:18.304834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:18.304473 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"d23cc47f9ea6a3d94587f0518d45fa1b1562b70b80291c3932b4f9343c60fd7a"} Jul 14 09:28:18.304834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:18.304504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8","Type":"ContainerStarted","Data":"d2c3ba52fe0cb59a0450a808ae17d5d0d824a6bfb0b42ee37c23138649a6e201"} Jul 14 09:28:18.333515 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:18.333474 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.551496893 podStartE2EDuration="13.333461365s" totalImagesPullingTime="9.752631012s" totalInitContainerRuntime="29.33346ms" isStatefulPod=false podCreationTimestamp="2026-07-14 09:28:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:28:18.332368573 +0000 UTC m=+109.270219044" watchObservedRunningTime="2026-07-14 09:28:18.333461365 +0000 UTC m=+109.271311834" Jul 14 09:28:21.136506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:21.136466 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:28:21.698599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:21.698561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:21.757976 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:21.757935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:21.811889 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:21.811854 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:24.600286 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:24.600249 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:24.674689 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:24.674655 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:29.349697 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:29.349662 2579 generic.go:356] "Generic (PLEG): container finished" podID="76baa4da-b647-4d01-81b8-9ce796611b65" containerID="0545d0b6af706d991723e6d8594275c8a9955461fa3a08d9b6dc077f71afdf07" exitCode=0 Jul 14 09:28:29.350117 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:29.349734 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" event={"ID":"76baa4da-b647-4d01-81b8-9ce796611b65","Type":"ContainerDied","Data":"0545d0b6af706d991723e6d8594275c8a9955461fa3a08d9b6dc077f71afdf07"} Jul 14 09:28:29.350117 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:29.350084 2579 scope.go:117] "RemoveContainer" containerID="0545d0b6af706d991723e6d8594275c8a9955461fa3a08d9b6dc077f71afdf07" Jul 14 09:28:30.355576 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:30.355538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hpqqg" event={"ID":"76baa4da-b647-4d01-81b8-9ce796611b65","Type":"ContainerStarted","Data":"991c9223f339e8eda744cf83233a24e4edb1c9f4b17f3d7f19b048a67226ffe8"} Jul 14 09:28:32.631487 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:32.631456 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:32.690267 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:32.690238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:32.795977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:32.795919 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" podUID="be49dc80-58a0-48ae-851d-42ac05324d71" containerName="registry" containerID="cri-o://bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a" gracePeriod=30 Jul 14 09:28:33.037873 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.037849 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:28:33.176970 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.176884 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-registry-certificates\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.176970 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.176933 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-image-registry-private-configuration\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.176970 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.176950 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-trusted-ca\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.176970 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.176966 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be49dc80-58a0-48ae-851d-42ac05324d71-ca-trust-extracted\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.177274 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.176986 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-bound-sa-token\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.177274 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177016 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-installation-pull-secrets\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.177274 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177036 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg2q4\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-kube-api-access-vg2q4\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.177274 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") pod \"be49dc80-58a0-48ae-851d-42ac05324d71\" (UID: \"be49dc80-58a0-48ae-851d-42ac05324d71\") " Jul 14 09:28:33.177471 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177373 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:28:33.177471 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177385 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:28:33.177789 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177742 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-registry-certificates\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.177916 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.177792 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be49dc80-58a0-48ae-851d-42ac05324d71-trusted-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.179814 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.179731 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 09:28:33.179814 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.179736 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-kube-api-access-vg2q4" (OuterVolumeSpecName: "kube-api-access-vg2q4") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "kube-api-access-vg2q4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:28:33.179959 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.179866 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:28:33.179959 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.179890 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 09:28:33.179959 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.179939 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:28:33.186037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.186010 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be49dc80-58a0-48ae-851d-42ac05324d71-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "be49dc80-58a0-48ae-851d-42ac05324d71" (UID: "be49dc80-58a0-48ae-851d-42ac05324d71"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:28:33.282988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.279134 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-image-registry-private-configuration\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.282988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.279176 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be49dc80-58a0-48ae-851d-42ac05324d71-ca-trust-extracted\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.282988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.279194 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-bound-sa-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.282988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.279219 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be49dc80-58a0-48ae-851d-42ac05324d71-installation-pull-secrets\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.282988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.279235 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg2q4\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-kube-api-access-vg2q4\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.282988 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.279249 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be49dc80-58a0-48ae-851d-42ac05324d71-registry-tls\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:28:33.367400 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.367363 2579 generic.go:356] "Generic (PLEG): container finished" podID="be49dc80-58a0-48ae-851d-42ac05324d71" containerID="bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a" exitCode=0 Jul 14 09:28:33.367559 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.367416 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" Jul 14 09:28:33.367559 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.367430 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" event={"ID":"be49dc80-58a0-48ae-851d-42ac05324d71","Type":"ContainerDied","Data":"bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a"} Jul 14 09:28:33.367559 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.367499 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fc449dc4-ft7gq" event={"ID":"be49dc80-58a0-48ae-851d-42ac05324d71","Type":"ContainerDied","Data":"a3ddbb85cfda33340dbcde892f44b9a8e19393aeb90a627cc488eebf476b108e"} Jul 14 09:28:33.367559 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.367522 2579 scope.go:117] "RemoveContainer" containerID="bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a" Jul 14 09:28:33.375688 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.375672 2579 scope.go:117] "RemoveContainer" containerID="bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a" Jul 14 09:28:33.375981 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:28:33.375949 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a\": container with ID starting with bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a not found: ID does not exist" containerID="bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a" Jul 14 09:28:33.376032 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.375990 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a"} err="failed to get container status \"bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a\": rpc error: code = NotFound desc = could not find container \"bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a\": container with ID starting with bf87d4b127695a8d26064b95727d6aff83d5c529fecf3da2cfd031bec5e28f3a not found: ID does not exist" Jul 14 09:28:33.389231 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.389209 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76fc449dc4-ft7gq"] Jul 14 09:28:33.396222 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.396201 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76fc449dc4-ft7gq"] Jul 14 09:28:33.635802 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:33.635762 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be49dc80-58a0-48ae-851d-42ac05324d71" path="/var/lib/kubelet/pods/be49dc80-58a0-48ae-851d-42ac05324d71/volumes" Jul 14 09:28:34.372133 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:34.372099 2579 generic.go:356] "Generic (PLEG): container finished" podID="a5e16cd1-ac7f-419f-a206-ee417e748911" containerID="b5bd7b1a77baad1ecba2b540a643ad483dc50b6bac0ca8cbd539dd43faef7a30" exitCode=0 Jul 14 09:28:34.372382 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:34.372170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" event={"ID":"a5e16cd1-ac7f-419f-a206-ee417e748911","Type":"ContainerDied","Data":"b5bd7b1a77baad1ecba2b540a643ad483dc50b6bac0ca8cbd539dd43faef7a30"} Jul 14 09:28:34.372567 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:34.372545 2579 scope.go:117] "RemoveContainer" containerID="b5bd7b1a77baad1ecba2b540a643ad483dc50b6bac0ca8cbd539dd43faef7a30" Jul 14 09:28:35.012406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:35.012375 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d68dfd9cb-dkqs8_839ee682-38ac-41bd-9798-cef813d75122/router/0.log" Jul 14 09:28:35.018310 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:35.018290 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j9jpq_259e96dd-ba4d-4586-8680-ffc219915b18/serve-healthcheck-canary/0.log" Jul 14 09:28:35.378802 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:35.378703 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tkhdz" event={"ID":"a5e16cd1-ac7f-419f-a206-ee417e748911","Type":"ContainerStarted","Data":"0107adc068f2f7c00fe634c2b326e9e8bc3b257dd3b01c6c549f8a7766b280c1"} Jul 14 09:28:38.425653 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:38.425619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:28:38.427895 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:38.427875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bcf77f5-947b-4432-b408-425e3e0612a6-metrics-certs\") pod \"network-metrics-daemon-fpqzv\" (UID: \"1bcf77f5-947b-4432-b408-425e3e0612a6\") " pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:28:38.567700 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:38.567666 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frh48\"" Jul 14 09:28:38.575260 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:38.575231 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fpqzv" Jul 14 09:28:38.740850 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:38.740823 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fpqzv"] Jul 14 09:28:38.742933 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:28:38.742908 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bcf77f5_947b_4432_b408_425e3e0612a6.slice/crio-4eec27483e6039ff61e3dc9cd232295a5373ceb7c16db10bad0f7b667d9064cf WatchSource:0}: Error finding container 4eec27483e6039ff61e3dc9cd232295a5373ceb7c16db10bad0f7b667d9064cf: Status 404 returned error can't find the container with id 4eec27483e6039ff61e3dc9cd232295a5373ceb7c16db10bad0f7b667d9064cf Jul 14 09:28:39.395484 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:39.395441 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fpqzv" event={"ID":"1bcf77f5-947b-4432-b408-425e3e0612a6","Type":"ContainerStarted","Data":"4eec27483e6039ff61e3dc9cd232295a5373ceb7c16db10bad0f7b667d9064cf"} Jul 14 09:28:40.401703 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:40.401665 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fpqzv" event={"ID":"1bcf77f5-947b-4432-b408-425e3e0612a6","Type":"ContainerStarted","Data":"bf6871f1487340d002cb0effbb00c9491a42dc97d3f25c1d533cc696ab01ac75"} Jul 14 09:28:41.407022 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:41.406989 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fpqzv" event={"ID":"1bcf77f5-947b-4432-b408-425e3e0612a6","Type":"ContainerStarted","Data":"1dff81b338d6af8fb6ef3dcef7f1db70b9015ab38cd0790929e878d57be2c1f9"} Jul 14 09:28:41.424651 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:41.424599 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fpqzv" podStartSLOduration=130.98049684 podStartE2EDuration="2m12.424581127s" totalImagesPullingTime="1.444084287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:28:41.423711857 +0000 UTC m=+132.361562329" watchObservedRunningTime="2026-07-14 09:28:41.424581127 +0000 UTC m=+132.362431597" Jul 14 09:28:41.513357 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:41.513325 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:41.614658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:41.614623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:45.592361 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:45.592312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:45.693292 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:45.693244 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:45.758312 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:45.758278 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:49.080944 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:49.080909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:49.135534 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:49.135505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:28:49.445251 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:49.445169 2579 generic.go:356] "Generic (PLEG): container finished" podID="a7e096b9-8d20-49c2-b4ca-22041fb503fc" containerID="e50dc727fa86dcea3cb90670c35328d6e0ee6efc35df9cdbd70f9e3cf4fba9ca" exitCode=0 Jul 14 09:28:49.445251 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:49.445214 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" event={"ID":"a7e096b9-8d20-49c2-b4ca-22041fb503fc","Type":"ContainerDied","Data":"e50dc727fa86dcea3cb90670c35328d6e0ee6efc35df9cdbd70f9e3cf4fba9ca"} Jul 14 09:28:49.445543 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:49.445528 2579 scope.go:117] "RemoveContainer" containerID="e50dc727fa86dcea3cb90670c35328d6e0ee6efc35df9cdbd70f9e3cf4fba9ca" Jul 14 09:28:50.451812 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:28:50.451776 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-sxzmc" event={"ID":"a7e096b9-8d20-49c2-b4ca-22041fb503fc","Type":"ContainerStarted","Data":"7e095e15fa5feb8f26142b69dbbca65bfa88106369e45211d93ece1d6c82063e"} Jul 14 09:29:06.136444 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:06.136389 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:29:06.152396 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:06.152369 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:29:06.528483 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:06.528457 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 09:29:15.589790 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:15.589740 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:15.647632 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:15.647595 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:24.626856 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:24.626735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:24.698278 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:24.698231 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:27.534378 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:27.534339 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:27.599469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:27.599433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:35.964811 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:35.964776 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:36.068327 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:36.068287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:36.123960 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:36.123922 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:39.601943 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:39.601898 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:29:39.703251 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:29:39.703201 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:05.173458 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:05.173410 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:05.266226 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:05.266187 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:09.104845 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:09.104811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:09.205480 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:09.205446 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:19.024593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:19.024549 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:46.103547 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:46.103508 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:47.268176 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:47.268141 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:50.250204 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:50.250156 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:50.377627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:50.377591 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:54.696880 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:54.696794 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:54.855683 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:54.855645 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:55.056253 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:55.056215 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:55.264285 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:55.264249 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:55.860429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:55.860386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:55.983389 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:55.983344 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:57.274329 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:57.274272 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:30:57.752573 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:30:57.752482 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:03.836893 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:03.836848 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:03.937166 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:03.937133 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:05.203381 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:05.203328 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:08.101647 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:08.101616 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:08.198298 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:08.198255 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:14.100490 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:14.100455 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:14.200895 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:14.200860 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:15.977469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:15.977440 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:29.492311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:29.492283 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:31:29.495290 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:29.495261 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:31:29.504408 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:29.504388 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 09:31:31.965626 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:31.965586 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:32.061275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:32.061243 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:32.753883 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:32.753849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:32.993822 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:32.993782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.094186 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.094148 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.120989 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.120958 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv"] Jul 14 09:31:33.121282 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.121269 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be49dc80-58a0-48ae-851d-42ac05324d71" containerName="registry" Jul 14 09:31:33.121323 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.121284 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be49dc80-58a0-48ae-851d-42ac05324d71" containerName="registry" Jul 14 09:31:33.121363 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.121340 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be49dc80-58a0-48ae-851d-42ac05324d71" containerName="registry" Jul 14 09:31:33.124328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.124312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.126913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.126895 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 09:31:33.127035 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.126948 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 09:31:33.127035 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.126972 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 09:31:33.127035 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.126972 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vjhmj\"" Jul 14 09:31:33.127239 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.127221 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 09:31:33.137018 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.136995 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv"] Jul 14 09:31:33.185364 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.185327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-manifests\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.185364 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.185367 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxv8v\" (UniqueName: \"kubernetes.io/projected/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-kube-api-access-xxv8v\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.185597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.185492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-tmp\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.185597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.185518 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.185597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.185561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-webhook-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.281728 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.281697 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.286811 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.286775 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-tmp\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.286937 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.286824 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.286937 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.286876 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-webhook-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.286937 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.286912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-manifests\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.287093 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.286939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxv8v\" (UniqueName: \"kubernetes.io/projected/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-kube-api-access-xxv8v\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.287281 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.287259 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-manifests\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.289243 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.289218 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-tmp\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.289348 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.289219 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.289408 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.289370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-webhook-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.294551 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.294532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxv8v\" (UniqueName: \"kubernetes.io/projected/ce3867ae-c493-4a1a-8399-1cafabe0ef2d-kube-api-access-xxv8v\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-vlnbv\" (UID: \"ce3867ae-c493-4a1a-8399-1cafabe0ef2d\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.382103 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.381995 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.435120 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.435083 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:33.498515 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.498469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.582389 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.582234 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv"] Jul 14 09:31:33.585297 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:31:33.585266 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3867ae_c493_4a1a_8399_1cafabe0ef2d.slice/crio-046d7951e324820aae6ccb01a3a7bc54c7adcac9319c1b89f16ff662d45c60da WatchSource:0}: Error finding container 046d7951e324820aae6ccb01a3a7bc54c7adcac9319c1b89f16ff662d45c60da: Status 404 returned error can't find the container with id 046d7951e324820aae6ccb01a3a7bc54c7adcac9319c1b89f16ff662d45c60da Jul 14 09:31:33.587317 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.587296 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 09:31:33.595302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.595276 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.673031 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.672943 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.778701 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.778669 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:33.879312 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:33.879263 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:34.090962 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:34.090930 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" event={"ID":"ce3867ae-c493-4a1a-8399-1cafabe0ef2d","Type":"ContainerStarted","Data":"046d7951e324820aae6ccb01a3a7bc54c7adcac9319c1b89f16ff662d45c60da"} Jul 14 09:31:37.110573 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:37.110531 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" event={"ID":"ce3867ae-c493-4a1a-8399-1cafabe0ef2d","Type":"ContainerStarted","Data":"65aca1d1f39a11ddf5a6ee949bd2dcfffed92f274700afee6d15b60764f7599f"} Jul 14 09:31:37.645349 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:37.645313 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:37.776726 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:37.776688 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:38.115760 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:38.115712 2579 generic.go:356] "Generic (PLEG): container finished" podID="ce3867ae-c493-4a1a-8399-1cafabe0ef2d" containerID="65aca1d1f39a11ddf5a6ee949bd2dcfffed92f274700afee6d15b60764f7599f" exitCode=0 Jul 14 09:31:38.116221 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:38.115785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" event={"ID":"ce3867ae-c493-4a1a-8399-1cafabe0ef2d","Type":"ContainerDied","Data":"65aca1d1f39a11ddf5a6ee949bd2dcfffed92f274700afee6d15b60764f7599f"} Jul 14 09:31:39.121789 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:39.121735 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" event={"ID":"ce3867ae-c493-4a1a-8399-1cafabe0ef2d","Type":"ContainerStarted","Data":"0a7dea0f61ed5d8853f8e2ee5b8d3fb582537c92b68daecca3144ea43783dfe4"} Jul 14 09:31:39.122180 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:39.121869 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:39.154426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:39.154376 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" podStartSLOduration=1.759284265 podStartE2EDuration="6.154360278s" totalImagesPullingTime="3.356222145s" totalInitContainerRuntime="1.038853868s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:31:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:31:39.150120798 +0000 UTC m=+310.087971259" watchObservedRunningTime="2026-07-14 09:31:39.154360278 +0000 UTC m=+310.092210748" Jul 14 09:31:44.604736 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:44.604699 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:47.152859 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:47.152829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:50.127633 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:50.127604 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-vlnbv" Jul 14 09:31:52.259329 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:52.259287 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:52.392602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:52.392564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:52.935172 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:52.935135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:53.014464 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.014432 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc"] Jul 14 09:31:53.017911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.017887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.022614 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.022592 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ssjgj\"" Jul 14 09:31:53.022972 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.022957 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 09:31:53.023386 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.023364 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 09:31:53.045401 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.045368 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc"] Jul 14 09:31:53.164637 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.164594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/252393e4-ea9b-4a34-bc5e-e26d1281c146-tmp\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.164834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.164710 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.164834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.164763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4ph\" (UniqueName: \"kubernetes.io/projected/252393e4-ea9b-4a34-bc5e-e26d1281c146-kube-api-access-tf4ph\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.265946 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.265903 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.266425 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.265957 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4ph\" (UniqueName: \"kubernetes.io/projected/252393e4-ea9b-4a34-bc5e-e26d1281c146-kube-api-access-tf4ph\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.266425 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.266012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/252393e4-ea9b-4a34-bc5e-e26d1281c146-tmp\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.266425 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:31:53.266088 2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.266425 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:31:53.266176 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs podName:252393e4-ea9b-4a34-bc5e-e26d1281c146 nodeName:}" failed. No retries permitted until 2026-07-14 09:31:53.766157554 +0000 UTC m=+324.704008006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs") pod "kube-auth-proxy-8569779fcf-vwwcc" (UID: "252393e4-ea9b-4a34-bc5e-e26d1281c146") : secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.266425 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.266281 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pmw4g"] Jul 14 09:31:53.268674 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.268651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/252393e4-ea9b-4a34-bc5e-e26d1281c146-tmp\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.270000 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.269986 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.272408 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.272387 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2zjvz\"" Jul 14 09:31:53.272825 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.272807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 14 09:31:53.279800 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.279732 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4ph\" (UniqueName: \"kubernetes.io/projected/252393e4-ea9b-4a34-bc5e-e26d1281c146-kube-api-access-tf4ph\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.286442 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.286413 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pmw4g"] Jul 14 09:31:53.339507 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.339477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:53.367439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.367407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sg6t\" (UniqueName: \"kubernetes.io/projected/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-kube-api-access-8sg6t\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.367627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.367512 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-cert\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.468711 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.468676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-cert\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.468911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.468774 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sg6t\" (UniqueName: \"kubernetes.io/projected/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-kube-api-access-8sg6t\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.468911 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:31:53.468852 2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 14 09:31:53.468997 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:31:53.468923 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-cert podName:46ebccb9-1657-430b-ac2b-245b8ad3cfd1 nodeName:}" failed. No retries permitted until 2026-07-14 09:31:53.968908825 +0000 UTC m=+324.906759276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-cert") pod "odh-model-controller-7998c67dc4-pmw4g" (UID: "46ebccb9-1657-430b-ac2b-245b8ad3cfd1") : secret "odh-model-controller-webhook-cert" not found Jul 14 09:31:53.481061 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.481028 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sg6t\" (UniqueName: \"kubernetes.io/projected/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-kube-api-access-8sg6t\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.622900 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.622806 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:53.770986 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.770948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:53.771205 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.771068 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:53.771205 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:31:53.771083 2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.771205 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:31:53.771159 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs podName:252393e4-ea9b-4a34-bc5e-e26d1281c146 nodeName:}" failed. No retries permitted until 2026-07-14 09:31:54.771138881 +0000 UTC m=+325.708989331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs") pod "kube-auth-proxy-8569779fcf-vwwcc" (UID: "252393e4-ea9b-4a34-bc5e-e26d1281c146") : secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.972298 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.972220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-cert\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:53.974634 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:53.974602 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46ebccb9-1657-430b-ac2b-245b8ad3cfd1-cert\") pod \"odh-model-controller-7998c67dc4-pmw4g\" (UID: \"46ebccb9-1657-430b-ac2b-245b8ad3cfd1\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:54.023789 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.023731 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:54.199473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.199431 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:54.265015 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.264948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:54.353357 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.353322 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pmw4g"] Jul 14 09:31:54.356279 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:31:54.356242 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ebccb9_1657_430b_ac2b_245b8ad3cfd1.slice/crio-eb7cc6913151fa78dd9e30c936d96bf5f1661ed7d3c886d058a53c68f88f0bc2 WatchSource:0}: Error finding container eb7cc6913151fa78dd9e30c936d96bf5f1661ed7d3c886d058a53c68f88f0bc2: Status 404 returned error can't find the container with id eb7cc6913151fa78dd9e30c936d96bf5f1661ed7d3c886d058a53c68f88f0bc2 Jul 14 09:31:54.476183 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.476143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:54.565322 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.565283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:54.780395 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.780346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:54.782956 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.782931 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/252393e4-ea9b-4a34-bc5e-e26d1281c146-tls-certs\") pod \"kube-auth-proxy-8569779fcf-vwwcc\" (UID: \"252393e4-ea9b-4a34-bc5e-e26d1281c146\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:54.827128 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.827034 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" Jul 14 09:31:54.963464 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:54.963432 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc"] Jul 14 09:31:54.965937 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:31:54.965908 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252393e4_ea9b_4a34_bc5e_e26d1281c146.slice/crio-bfec0896658104dd605f00938b73b07cb73f70b6da6dfcde9168d5660ab1066f WatchSource:0}: Error finding container bfec0896658104dd605f00938b73b07cb73f70b6da6dfcde9168d5660ab1066f: Status 404 returned error can't find the container with id bfec0896658104dd605f00938b73b07cb73f70b6da6dfcde9168d5660ab1066f Jul 14 09:31:55.193229 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:55.193130 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" event={"ID":"252393e4-ea9b-4a34-bc5e-e26d1281c146","Type":"ContainerStarted","Data":"bfec0896658104dd605f00938b73b07cb73f70b6da6dfcde9168d5660ab1066f"} Jul 14 09:31:55.194338 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:55.194306 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" event={"ID":"46ebccb9-1657-430b-ac2b-245b8ad3cfd1","Type":"ContainerStarted","Data":"eb7cc6913151fa78dd9e30c936d96bf5f1661ed7d3c886d058a53c68f88f0bc2"} Jul 14 09:31:55.727235 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:55.727199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:58.212523 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.212391 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" event={"ID":"252393e4-ea9b-4a34-bc5e-e26d1281c146","Type":"ContainerStarted","Data":"24c0cccfebcf25f72f1748198414ce3330c022ec2330409f2338bc82dce76487"} Jul 14 09:31:58.214542 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.214461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" event={"ID":"46ebccb9-1657-430b-ac2b-245b8ad3cfd1","Type":"ContainerStarted","Data":"4ff1fda681a7867d1353b07d13802169220fd4d2472575abcf8d59f674c83eea"} Jul 14 09:31:58.214542 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.214531 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:31:58.245797 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.245721 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8569779fcf-vwwcc" podStartSLOduration=3.092676197 podStartE2EDuration="6.245702536s" totalImagesPullingTime="3.153026339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:31:58.244304971 +0000 UTC m=+329.182155442" watchObservedRunningTime="2026-07-14 09:31:58.245702536 +0000 UTC m=+329.183553006" Jul 14 09:31:58.288216 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.288163 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" podStartSLOduration=1.524874943 podStartE2EDuration="5.288141464s" totalImagesPullingTime="3.763266521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:31:58.286460361 +0000 UTC m=+329.224310834" watchObservedRunningTime="2026-07-14 09:31:58.288141464 +0000 UTC m=+329.225991933" Jul 14 09:31:58.599318 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.599278 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:31:58.743846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:31:58.743806 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:01.711718 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.711684 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7"] Jul 14 09:32:01.714634 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.714615 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.729656 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.729631 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 09:32:01.729836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.729716 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 09:32:01.729836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.729740 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 09:32:01.729836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.729784 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 09:32:01.731323 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.731303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7"] Jul 14 09:32:01.740489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.740466 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dz5h5\"" Jul 14 09:32:01.839605 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.839839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.839839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.839839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839710 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.839839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839821 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.840016 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swpzr\" (UniqueName: \"kubernetes.io/projected/a77b754d-9b25-4377-aa46-58ef2c3782c7-kube-api-access-swpzr\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.840016 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.839923 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77b754d-9b25-4377-aa46-58ef2c3782c7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.940808 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.940808 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.941037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.941037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swpzr\" (UniqueName: \"kubernetes.io/projected/a77b754d-9b25-4377-aa46-58ef2c3782c7-kube-api-access-swpzr\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.941037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77b754d-9b25-4377-aa46-58ef2c3782c7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.941037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.941037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.940998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.941602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.941578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.943350 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.943316 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.943470 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.943401 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.943535 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.943519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77b754d-9b25-4377-aa46-58ef2c3782c7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.943741 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.943723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.958825 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.958800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77b754d-9b25-4377-aa46-58ef2c3782c7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:01.969019 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:01.968962 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swpzr\" (UniqueName: \"kubernetes.io/projected/a77b754d-9b25-4377-aa46-58ef2c3782c7-kube-api-access-swpzr\") pod \"istiod-openshift-gateway-94bb8fbfd-p57p7\" (UID: \"a77b754d-9b25-4377-aa46-58ef2c3782c7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:02.024268 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:02.024230 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:02.201863 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:32:02.201809 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77b754d_9b25_4377_aa46_58ef2c3782c7.slice/crio-976d978e646c7109ff5b6e1a0bf6ef6aa7db8865a5639ec9d31f4d3e69e610d1 WatchSource:0}: Error finding container 976d978e646c7109ff5b6e1a0bf6ef6aa7db8865a5639ec9d31f4d3e69e610d1: Status 404 returned error can't find the container with id 976d978e646c7109ff5b6e1a0bf6ef6aa7db8865a5639ec9d31f4d3e69e610d1 Jul 14 09:32:02.220416 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:02.220347 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7"] Jul 14 09:32:02.233173 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:02.233136 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" event={"ID":"a77b754d-9b25-4377-aa46-58ef2c3782c7","Type":"ContainerStarted","Data":"976d978e646c7109ff5b6e1a0bf6ef6aa7db8865a5639ec9d31f4d3e69e610d1"} Jul 14 09:32:04.996872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:04.996818 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:32:04.997150 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:04.996915 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:32:05.217022 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.216985 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r"] Jul 14 09:32:05.220610 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.220583 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.250144 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.250059 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r"] Jul 14 09:32:05.252703 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.252673 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" event={"ID":"a77b754d-9b25-4377-aa46-58ef2c3782c7","Type":"ContainerStarted","Data":"01e0584841acf7364bdcd2ae7f8e88d045bee4c633f3dd687582c1a1967032f1"} Jul 14 09:32:05.252889 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.252872 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:05.257288 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.257269 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ps4qt\"" Jul 14 09:32:05.373023 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.372991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzhxj\" (UniqueName: \"kubernetes.io/projected/92b5df85-d783-4525-969c-508b1b5612af-kube-api-access-pzhxj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373023 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373028 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373305 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373305 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373218 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373305 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373256 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b5df85-d783-4525-969c-508b1b5612af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373472 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b5df85-d783-4525-969c-508b1b5612af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373472 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373436 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b5df85-d783-4525-969c-508b1b5612af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373592 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373473 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.373592 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.373539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.405642 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.405580 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" podStartSLOduration=1.612872598 podStartE2EDuration="4.405561693s" totalImagesPullingTime="2.792689095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:32:05.397475086 +0000 UTC m=+336.335325557" watchObservedRunningTime="2026-07-14 09:32:05.405561693 +0000 UTC m=+336.343412163" Jul 14 09:32:05.474764 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474722 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.474954 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.474954 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzhxj\" (UniqueName: \"kubernetes.io/projected/92b5df85-d783-4525-969c-508b1b5612af-kube-api-access-pzhxj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.474954 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.474954 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.474954 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.474954 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b5df85-d783-4525-969c-508b1b5612af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475278 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.474974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b5df85-d783-4525-969c-508b1b5612af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475278 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.475022 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b5df85-d783-4525-969c-508b1b5612af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475278 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.475206 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475422 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.475372 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475422 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.475417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.475633 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.475742 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.475717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b5df85-d783-4525-969c-508b1b5612af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.477369 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.477341 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b5df85-d783-4525-969c-508b1b5612af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.477627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.477605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b5df85-d783-4525-969c-508b1b5612af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.502010 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.501934 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b5df85-d783-4525-969c-508b1b5612af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.502010 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.501999 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzhxj\" (UniqueName: \"kubernetes.io/projected/92b5df85-d783-4525-969c-508b1b5612af-kube-api-access-pzhxj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s4n6r\" (UID: \"92b5df85-d783-4525-969c-508b1b5612af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.533912 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.533874 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:05.578204 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.578164 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:05.796342 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:32:05.796315 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b5df85_d783_4525_969c_508b1b5612af.slice/crio-101d2e30b9c33cce0aeaab3ca462381290ebd26551f9ffe18e645fdae17ec83b WatchSource:0}: Error finding container 101d2e30b9c33cce0aeaab3ca462381290ebd26551f9ffe18e645fdae17ec83b: Status 404 returned error can't find the container with id 101d2e30b9c33cce0aeaab3ca462381290ebd26551f9ffe18e645fdae17ec83b Jul 14 09:32:05.818462 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.818425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:05.822277 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:05.822254 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r"] Jul 14 09:32:06.026586 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.026542 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:06.132885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.132767 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:06.169663 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.169630 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj"] Jul 14 09:32:06.174280 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.174255 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.185772 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.185729 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj"] Jul 14 09:32:06.199695 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.199665 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 14 09:32:06.199695 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.199668 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jrlzm\"" Jul 14 09:32:06.219607 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.219555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:06.222405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.222381 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7"] Jul 14 09:32:06.225625 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.225601 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.236684 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.236657 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 14 09:32:06.246111 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.246076 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-q8w2b\"" Jul 14 09:32:06.259658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.259625 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" event={"ID":"92b5df85-d783-4525-969c-508b1b5612af","Type":"ContainerStarted","Data":"101d2e30b9c33cce0aeaab3ca462381290ebd26551f9ffe18e645fdae17ec83b"} Jul 14 09:32:06.261562 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.261536 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-p57p7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 09:32:06.261725 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.261585 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" podUID="a77b754d-9b25-4377-aa46-58ef2c3782c7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 09:32:06.279630 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.276694 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7"] Jul 14 09:32:06.283162 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.283138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4tmf\" (UniqueName: \"kubernetes.io/projected/bcbfca45-4486-46da-be18-58947d55bbc6-kube-api-access-q4tmf\") pod \"kserve-controller-manager-d6f4b65b8-c2cfj\" (UID: \"bcbfca45-4486-46da-be18-58947d55bbc6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.283270 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.283220 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bcbfca45-4486-46da-be18-58947d55bbc6-cert\") pod \"kserve-controller-manager-d6f4b65b8-c2cfj\" (UID: \"bcbfca45-4486-46da-be18-58947d55bbc6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.384738 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.384639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d21a6b72-b030-427b-a39b-285ac161fdb3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4qwz7\" (UID: \"d21a6b72-b030-427b-a39b-285ac161fdb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.384959 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.384861 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4tmf\" (UniqueName: \"kubernetes.io/projected/bcbfca45-4486-46da-be18-58947d55bbc6-kube-api-access-q4tmf\") pod \"kserve-controller-manager-d6f4b65b8-c2cfj\" (UID: \"bcbfca45-4486-46da-be18-58947d55bbc6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.384959 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.384948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kbbx\" (UniqueName: \"kubernetes.io/projected/d21a6b72-b030-427b-a39b-285ac161fdb3-kube-api-access-4kbbx\") pod \"llmisvc-controller-manager-777c6bd7d5-4qwz7\" (UID: \"d21a6b72-b030-427b-a39b-285ac161fdb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.385053 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.384986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bcbfca45-4486-46da-be18-58947d55bbc6-cert\") pod \"kserve-controller-manager-d6f4b65b8-c2cfj\" (UID: \"bcbfca45-4486-46da-be18-58947d55bbc6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.387409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.387390 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bcbfca45-4486-46da-be18-58947d55bbc6-cert\") pod \"kserve-controller-manager-d6f4b65b8-c2cfj\" (UID: \"bcbfca45-4486-46da-be18-58947d55bbc6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.403185 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.403153 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4tmf\" (UniqueName: \"kubernetes.io/projected/bcbfca45-4486-46da-be18-58947d55bbc6-kube-api-access-q4tmf\") pod \"kserve-controller-manager-d6f4b65b8-c2cfj\" (UID: \"bcbfca45-4486-46da-be18-58947d55bbc6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.453240 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.453202 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:06.485974 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.485939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbbx\" (UniqueName: \"kubernetes.io/projected/d21a6b72-b030-427b-a39b-285ac161fdb3-kube-api-access-4kbbx\") pod \"llmisvc-controller-manager-777c6bd7d5-4qwz7\" (UID: \"d21a6b72-b030-427b-a39b-285ac161fdb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.486127 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.486028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d21a6b72-b030-427b-a39b-285ac161fdb3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4qwz7\" (UID: \"d21a6b72-b030-427b-a39b-285ac161fdb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.486772 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.486740 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:06.488477 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.488459 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d21a6b72-b030-427b-a39b-285ac161fdb3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4qwz7\" (UID: \"d21a6b72-b030-427b-a39b-285ac161fdb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.506804 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.506767 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kbbx\" (UniqueName: \"kubernetes.io/projected/d21a6b72-b030-427b-a39b-285ac161fdb3-kube-api-access-4kbbx\") pod \"llmisvc-controller-manager-777c6bd7d5-4qwz7\" (UID: \"d21a6b72-b030-427b-a39b-285ac161fdb3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.538512 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.538398 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:06.588140 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.588095 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:06.655313 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:32:06.655280 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbfca45_4486_46da_be18_58947d55bbc6.slice/crio-d2640dc91ba1c2395dbb63ba5967fdc5a071f5fecd744b7ddedf9339e2b2f42c WatchSource:0}: Error finding container d2640dc91ba1c2395dbb63ba5967fdc5a071f5fecd744b7ddedf9339e2b2f42c: Status 404 returned error can't find the container with id d2640dc91ba1c2395dbb63ba5967fdc5a071f5fecd744b7ddedf9339e2b2f42c Jul 14 09:32:06.664119 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.664090 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj"] Jul 14 09:32:06.710671 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:32:06.710644 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd21a6b72_b030_427b_a39b_285ac161fdb3.slice/crio-3e3db5d57fb50b45243f6259be4da2d99384b8fe176d4dc25257747d4304e797 WatchSource:0}: Error finding container 3e3db5d57fb50b45243f6259be4da2d99384b8fe176d4dc25257747d4304e797: Status 404 returned error can't find the container with id 3e3db5d57fb50b45243f6259be4da2d99384b8fe176d4dc25257747d4304e797 Jul 14 09:32:06.715201 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:06.715176 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7"] Jul 14 09:32:07.266261 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.266224 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" event={"ID":"d21a6b72-b030-427b-a39b-285ac161fdb3","Type":"ContainerStarted","Data":"3e3db5d57fb50b45243f6259be4da2d99384b8fe176d4dc25257747d4304e797"} Jul 14 09:32:07.267649 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.267618 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" event={"ID":"bcbfca45-4486-46da-be18-58947d55bbc6","Type":"ContainerStarted","Data":"d2640dc91ba1c2395dbb63ba5967fdc5a071f5fecd744b7ddedf9339e2b2f42c"} Jul 14 09:32:07.507873 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.507793 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:07.621083 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.620904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:07.718786 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.718676 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:07.880169 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.880056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:07.980989 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:07.980906 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:09.221914 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:09.221883 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pmw4g" Jul 14 09:32:09.260791 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:09.260724 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p57p7" Jul 14 09:32:09.834259 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:09.834212 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:32:09.834406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:09.834301 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:32:09.834406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:09.834333 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:32:11.290691 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.290652 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" event={"ID":"d21a6b72-b030-427b-a39b-285ac161fdb3","Type":"ContainerStarted","Data":"c90ab6d6b5fdd19d07f0e4b03f4b41eb045b653662d95943a3cd5f7a40600c24"} Jul 14 09:32:11.291212 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.290804 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:11.292373 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.292351 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" event={"ID":"92b5df85-d783-4525-969c-508b1b5612af","Type":"ContainerStarted","Data":"a328a4f5c6c7e7f50a6795f70c8d744b7fff30cefb1a1a4e2e3a80d01d7a020b"} Jul 14 09:32:11.293897 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.293876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" event={"ID":"bcbfca45-4486-46da-be18-58947d55bbc6","Type":"ContainerStarted","Data":"a29603d438675bae20e25861d06f5e7a106d214ec2ec87034c250752f80bd59a"} Jul 14 09:32:11.293973 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.293928 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:11.324545 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.324487 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" podStartSLOduration=1.591166852 podStartE2EDuration="5.324470191s" totalImagesPullingTime="3.733303339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:32:11.319912181 +0000 UTC m=+342.257762652" watchObservedRunningTime="2026-07-14 09:32:11.324470191 +0000 UTC m=+342.262320661" Jul 14 09:32:11.352220 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.352164 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" podStartSLOduration=2.259685337 podStartE2EDuration="5.352148222s" totalImagesPullingTime="3.092462885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:32:11.351880555 +0000 UTC m=+342.289731035" watchObservedRunningTime="2026-07-14 09:32:11.352148222 +0000 UTC m=+342.289998691" Jul 14 09:32:11.388403 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.388352 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" podStartSLOduration=2.353599779 podStartE2EDuration="6.388333705s" totalImagesPullingTime="4.034733926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:32:11.384921923 +0000 UTC m=+342.322772393" watchObservedRunningTime="2026-07-14 09:32:11.388333705 +0000 UTC m=+342.326184174" Jul 14 09:32:11.534071 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.534041 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:11.538775 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:11.538728 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:12.299670 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:12.299637 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:12.300777 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:12.300756 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s4n6r" Jul 14 09:32:16.146057 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:16.146018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:16.200313 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:16.200274 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:42.301767 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:42.301642 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4qwz7" Jul 14 09:32:42.304623 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:42.304606 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2cfj" Jul 14 09:32:51.109306 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:51.109266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:53.844666 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:53.844631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:54.153507 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:54.153425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:32:54.224008 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:32:54.223972 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:05.696327 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:05.696290 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:07.640241 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:07.640204 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:07.704500 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:07.704460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:10.587721 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:10.587685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:10.650469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:10.650437 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:11.167526 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:11.167488 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:11.445762 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:11.445663 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:21.136194 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:21.136152 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:21.244627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:21.244589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:21.391714 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:21.391600 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:22.363696 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:22.363664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:25.799818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:25.799784 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:27.361553 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:27.361499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:27.767283 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:27.767241 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:28.882242 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:28.882206 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:30.456597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:30.456566 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:30.560863 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:30.560824 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:31.927627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.927591 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv"] Jul 14 09:33:31.931006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.930983 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:31.933792 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.933774 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qlpkw\"" Jul 14 09:33:31.934731 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.934717 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 09:33:31.934865 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.934731 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 09:33:31.934865 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.934720 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 14 09:33:31.944635 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:31.944612 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv"] Jul 14 09:33:32.054719 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.054683 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n8ll\" (UniqueName: \"kubernetes.io/projected/a5f18558-cd42-4681-8f58-46c4c08bea7d-kube-api-access-7n8ll\") pod \"dns-operator-controller-manager-864dd78c4c-dqpnv\" (UID: \"a5f18558-cd42-4681-8f58-46c4c08bea7d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:32.156113 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.156078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n8ll\" (UniqueName: \"kubernetes.io/projected/a5f18558-cd42-4681-8f58-46c4c08bea7d-kube-api-access-7n8ll\") pod \"dns-operator-controller-manager-864dd78c4c-dqpnv\" (UID: \"a5f18558-cd42-4681-8f58-46c4c08bea7d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:32.172007 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.171980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n8ll\" (UniqueName: \"kubernetes.io/projected/a5f18558-cd42-4681-8f58-46c4c08bea7d-kube-api-access-7n8ll\") pod \"dns-operator-controller-manager-864dd78c4c-dqpnv\" (UID: \"a5f18558-cd42-4681-8f58-46c4c08bea7d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:32.241887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.241781 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:32.383089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.383055 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv"] Jul 14 09:33:32.385308 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:33:32.385273 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f18558_cd42_4681_8f58_46c4c08bea7d.slice/crio-e027f4684365e403895de54892121ff40a0ec9a9ca8c69479e591531cfa69cf1 WatchSource:0}: Error finding container e027f4684365e403895de54892121ff40a0ec9a9ca8c69479e591531cfa69cf1: Status 404 returned error can't find the container with id e027f4684365e403895de54892121ff40a0ec9a9ca8c69479e591531cfa69cf1 Jul 14 09:33:32.570801 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.570731 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:32.661002 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:32.660966 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" event={"ID":"a5f18558-cd42-4681-8f58-46c4c08bea7d","Type":"ContainerStarted","Data":"e027f4684365e403895de54892121ff40a0ec9a9ca8c69479e591531cfa69cf1"} Jul 14 09:33:34.071987 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.071954 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb"] Jul 14 09:33:34.075235 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.075207 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:34.077885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.077865 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fkrt4\"" Jul 14 09:33:34.087638 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.087612 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb"] Jul 14 09:33:34.176894 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.176849 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cbln\" (UniqueName: \"kubernetes.io/projected/72de3d55-94eb-4f60-81e9-6a6cc34bb023-kube-api-access-7cbln\") pod \"limitador-operator-controller-manager-859d7b78bd-w79qb\" (UID: \"72de3d55-94eb-4f60-81e9-6a6cc34bb023\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:34.278222 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.278175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cbln\" (UniqueName: \"kubernetes.io/projected/72de3d55-94eb-4f60-81e9-6a6cc34bb023-kube-api-access-7cbln\") pod \"limitador-operator-controller-manager-859d7b78bd-w79qb\" (UID: \"72de3d55-94eb-4f60-81e9-6a6cc34bb023\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:34.292351 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.292321 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cbln\" (UniqueName: \"kubernetes.io/projected/72de3d55-94eb-4f60-81e9-6a6cc34bb023-kube-api-access-7cbln\") pod \"limitador-operator-controller-manager-859d7b78bd-w79qb\" (UID: \"72de3d55-94eb-4f60-81e9-6a6cc34bb023\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:34.385302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.385202 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:34.415498 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.415456 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:34.845494 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:34.845474 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb"] Jul 14 09:33:34.847666 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:33:34.847640 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72de3d55_94eb_4f60_81e9_6a6cc34bb023.slice/crio-99f4f87fe8a0316bb59d2263babbcd2f33d15a531dca9f6b82a478b6f684b17e WatchSource:0}: Error finding container 99f4f87fe8a0316bb59d2263babbcd2f33d15a531dca9f6b82a478b6f684b17e: Status 404 returned error can't find the container with id 99f4f87fe8a0316bb59d2263babbcd2f33d15a531dca9f6b82a478b6f684b17e Jul 14 09:33:35.681313 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:35.681271 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" event={"ID":"a5f18558-cd42-4681-8f58-46c4c08bea7d","Type":"ContainerStarted","Data":"4607adf8eea4fa6063e7da32e010c870057ccf0db96c4760589ba5024a38af6c"} Jul 14 09:33:35.681742 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:35.681345 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:35.682649 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:35.682621 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" event={"ID":"72de3d55-94eb-4f60-81e9-6a6cc34bb023","Type":"ContainerStarted","Data":"99f4f87fe8a0316bb59d2263babbcd2f33d15a531dca9f6b82a478b6f684b17e"} Jul 14 09:33:35.708311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:35.708254 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" podStartSLOduration=2.339220228 podStartE2EDuration="4.708238574s" totalImagesPullingTime="2.369018346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:33:35.708029491 +0000 UTC m=+426.645879961" watchObservedRunningTime="2026-07-14 09:33:35.708238574 +0000 UTC m=+426.646089044" Jul 14 09:33:36.175740 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:36.175703 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:36.249183 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:36.249140 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:36.912089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:36.912047 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:37.120572 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:37.120529 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:37.695193 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:37.695156 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" event={"ID":"72de3d55-94eb-4f60-81e9-6a6cc34bb023","Type":"ContainerStarted","Data":"19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7"} Jul 14 09:33:37.695373 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:37.695232 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:37.713249 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:37.713197 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" podStartSLOduration=1.6088057180000002 podStartE2EDuration="3.713181247s" totalImagesPullingTime="2.104375529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:33:37.712128276 +0000 UTC m=+428.649978745" watchObservedRunningTime="2026-07-14 09:33:37.713181247 +0000 UTC m=+428.651031717" Jul 14 09:33:45.960668 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:45.960634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:46.053186 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:46.053142 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:46.603204 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:46.603172 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:46.692930 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:46.692904 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dqpnv" Jul 14 09:33:46.840256 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:46.840223 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:48.311961 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.311928 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb"] Jul 14 09:33:48.312431 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.312118 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" containerName="manager" containerID="cri-o://19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7" gracePeriod=2 Jul 14 09:33:48.314183 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.314159 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.323475 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.323442 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb"] Jul 14 09:33:48.333825 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.333793 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.363527 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.363496 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx"] Jul 14 09:33:48.363870 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.363856 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" containerName="manager" Jul 14 09:33:48.363923 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.363871 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" containerName="manager" Jul 14 09:33:48.363969 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.363942 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" containerName="manager" Jul 14 09:33:48.366811 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.366795 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:33:48.380303 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.380271 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx"] Jul 14 09:33:48.424560 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.424509 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.473685 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.473652 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s"] Jul 14 09:33:48.477037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.477018 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.479666 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.479632 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.480495 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.480472 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l4nht\"" Jul 14 09:33:48.494524 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.494437 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s"] Jul 14 09:33:48.516532 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.516294 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2thg\" (UniqueName: \"kubernetes.io/projected/04458047-5624-4d44-84a0-931854062395-kube-api-access-p2thg\") pod \"limitador-operator-controller-manager-859d7b78bd-fdvvx\" (UID: \"04458047-5624-4d44-84a0-931854062395\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:33:48.576206 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.576151 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.578381 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.578355 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.617506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.617472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmsvz\" (UniqueName: \"kubernetes.io/projected/f0822083-5720-4ec4-9d57-4c1917b8450e-kube-api-access-vmsvz\") pod \"kuadrant-operator-controller-manager-86878c769f-h5w6s\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.617658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.617609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2thg\" (UniqueName: \"kubernetes.io/projected/04458047-5624-4d44-84a0-931854062395-kube-api-access-p2thg\") pod \"limitador-operator-controller-manager-859d7b78bd-fdvvx\" (UID: \"04458047-5624-4d44-84a0-931854062395\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:33:48.617658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.617639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0822083-5720-4ec4-9d57-4c1917b8450e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h5w6s\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.629842 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.629808 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2thg\" (UniqueName: \"kubernetes.io/projected/04458047-5624-4d44-84a0-931854062395-kube-api-access-p2thg\") pod \"limitador-operator-controller-manager-859d7b78bd-fdvvx\" (UID: \"04458047-5624-4d44-84a0-931854062395\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:33:48.703491 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.703462 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:33:48.718684 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.718637 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cbln\" (UniqueName: \"kubernetes.io/projected/72de3d55-94eb-4f60-81e9-6a6cc34bb023-kube-api-access-7cbln\") pod \"72de3d55-94eb-4f60-81e9-6a6cc34bb023\" (UID: \"72de3d55-94eb-4f60-81e9-6a6cc34bb023\") " Jul 14 09:33:48.718871 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.718847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0822083-5720-4ec4-9d57-4c1917b8450e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h5w6s\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.718953 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.718907 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmsvz\" (UniqueName: \"kubernetes.io/projected/f0822083-5720-4ec4-9d57-4c1917b8450e-kube-api-access-vmsvz\") pod \"kuadrant-operator-controller-manager-86878c769f-h5w6s\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.719351 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.719326 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0822083-5720-4ec4-9d57-4c1917b8450e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h5w6s\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.726366 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.721325 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72de3d55-94eb-4f60-81e9-6a6cc34bb023-kube-api-access-7cbln" (OuterVolumeSpecName: "kube-api-access-7cbln") pod "72de3d55-94eb-4f60-81e9-6a6cc34bb023" (UID: "72de3d55-94eb-4f60-81e9-6a6cc34bb023"). InnerVolumeSpecName "kube-api-access-7cbln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:33:48.730934 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.730907 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmsvz\" (UniqueName: \"kubernetes.io/projected/f0822083-5720-4ec4-9d57-4c1917b8450e-kube-api-access-vmsvz\") pod \"kuadrant-operator-controller-manager-86878c769f-h5w6s\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.746157 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.745621 2579 generic.go:356] "Generic (PLEG): container finished" podID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" containerID="19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7" exitCode=0 Jul 14 09:33:48.746157 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.745684 2579 scope.go:117] "RemoveContainer" containerID="19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7" Jul 14 09:33:48.746157 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.745884 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.748648 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.748609 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.755681 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.755640 2579 scope.go:117] "RemoveContainer" containerID="19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7" Jul 14 09:33:48.756188 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.756156 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7\": container with ID starting with 19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7 not found: ID does not exist" containerID="19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7" Jul 14 09:33:48.756282 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.756200 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7"} err="failed to get container status \"19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7\": rpc error: code = NotFound desc = could not find container \"19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7\": container with ID starting with 19ac7558d08e8616d7dcc42480783fefe8e8e14f492b582d65e3de0d02d2e0f7 not found: ID does not exist" Jul 14 09:33:48.757932 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:48.757862 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:48.787350 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.787320 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:48.820182 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.820151 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cbln\" (UniqueName: \"kubernetes.io/projected/72de3d55-94eb-4f60-81e9-6a6cc34bb023-kube-api-access-7cbln\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:33:48.847997 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.847961 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx"] Jul 14 09:33:48.852810 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:33:48.852780 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04458047_5624_4d44_84a0_931854062395.slice/crio-355dcbc518a08f12a214ccb21220004f98311d3b4b8ade7b39cc6d281947812a WatchSource:0}: Error finding container 355dcbc518a08f12a214ccb21220004f98311d3b4b8ade7b39cc6d281947812a: Status 404 returned error can't find the container with id 355dcbc518a08f12a214ccb21220004f98311d3b4b8ade7b39cc6d281947812a Jul 14 09:33:48.935586 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:48.935559 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s"] Jul 14 09:33:48.936955 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:33:48.936934 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0822083_5720_4ec4_9d57_4c1917b8450e.slice/crio-425df9de305d285d353a178267d6c27a43cf21c8de21bf6eb1bd57e97b388e02 WatchSource:0}: Error finding container 425df9de305d285d353a178267d6c27a43cf21c8de21bf6eb1bd57e97b388e02: Status 404 returned error can't find the container with id 425df9de305d285d353a178267d6c27a43cf21c8de21bf6eb1bd57e97b388e02 Jul 14 09:33:49.636199 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:33:49.636097 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w79qb\" is forbidden: User \"system:node:ip-10-0-139-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-42.ec2.internal' and this object" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w79qb" Jul 14 09:33:49.639267 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:49.639240 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72de3d55-94eb-4f60-81e9-6a6cc34bb023" path="/var/lib/kubelet/pods/72de3d55-94eb-4f60-81e9-6a6cc34bb023/volumes" Jul 14 09:33:49.754844 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:49.754808 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" event={"ID":"f0822083-5720-4ec4-9d57-4c1917b8450e","Type":"ContainerStarted","Data":"425df9de305d285d353a178267d6c27a43cf21c8de21bf6eb1bd57e97b388e02"} Jul 14 09:33:49.756328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:49.756300 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" event={"ID":"04458047-5624-4d44-84a0-931854062395","Type":"ContainerStarted","Data":"d543b1ff7dc31eefd360c537e90363274d473c3e15d464ef952bbb4094bf440e"} Jul 14 09:33:49.756425 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:49.756343 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" event={"ID":"04458047-5624-4d44-84a0-931854062395","Type":"ContainerStarted","Data":"355dcbc518a08f12a214ccb21220004f98311d3b4b8ade7b39cc6d281947812a"} Jul 14 09:33:49.756425 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:49.756406 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:33:49.773351 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:49.773304 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" podStartSLOduration=1.773287662 podStartE2EDuration="1.773287662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:33:49.771863119 +0000 UTC m=+440.709713601" watchObservedRunningTime="2026-07-14 09:33:49.773287662 +0000 UTC m=+440.711138132" Jul 14 09:33:50.377388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:50.377348 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:50.480796 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:50.480742 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:54.363614 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:54.363574 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:54.472932 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:54.472897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:54.785175 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:54.785140 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" event={"ID":"f0822083-5720-4ec4-9d57-4c1917b8450e","Type":"ContainerStarted","Data":"ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9"} Jul 14 09:33:54.785356 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:54.785296 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:33:54.805400 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:54.805346 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" podStartSLOduration=1.8304252760000002 podStartE2EDuration="6.805327438s" totalImagesPullingTime="4.974902162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:33:54.803402171 +0000 UTC m=+445.741252641" watchObservedRunningTime="2026-07-14 09:33:54.805327438 +0000 UTC m=+445.743177971" Jul 14 09:33:58.974586 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:58.974555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:33:59.040304 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:33:59.040269 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:00.766228 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:00.766202 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdvvx" Jul 14 09:34:05.792603 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:05.792569 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:34:09.069763 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:09.069684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:09.170122 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:09.170086 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:09.269925 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:09.269891 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:10.814379 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:10.814345 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:10.941241 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:10.941210 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:15.571275 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:15.571233 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:15.673257 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:15.673220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:19.579603 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:19.579558 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:19.638714 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:19.638679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:22.302069 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:22.302032 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:22.459678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:22.459645 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:26.492199 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.492163 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:26.596474 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.596427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:26.652201 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.652165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:26.948583 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.948549 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:34:26.952454 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.952438 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:26.955044 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.955023 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qtnl9\"" Jul 14 09:34:26.955215 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.955172 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 09:34:26.974911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:26.974882 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:34:27.029521 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.029482 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:34:27.067492 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.067455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ca1f5c56-5803-4889-a36f-4190771c3e63-config-file\") pod \"limitador-limitador-54d6d988d5-4xh6w\" (UID: \"ca1f5c56-5803-4889-a36f-4190771c3e63\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.067661 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.067573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j6hr\" (UniqueName: \"kubernetes.io/projected/ca1f5c56-5803-4889-a36f-4190771c3e63-kube-api-access-5j6hr\") pod \"limitador-limitador-54d6d988d5-4xh6w\" (UID: \"ca1f5c56-5803-4889-a36f-4190771c3e63\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.168539 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.168503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j6hr\" (UniqueName: \"kubernetes.io/projected/ca1f5c56-5803-4889-a36f-4190771c3e63-kube-api-access-5j6hr\") pod \"limitador-limitador-54d6d988d5-4xh6w\" (UID: \"ca1f5c56-5803-4889-a36f-4190771c3e63\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.168703 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.168567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ca1f5c56-5803-4889-a36f-4190771c3e63-config-file\") pod \"limitador-limitador-54d6d988d5-4xh6w\" (UID: \"ca1f5c56-5803-4889-a36f-4190771c3e63\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.169212 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.169192 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ca1f5c56-5803-4889-a36f-4190771c3e63-config-file\") pod \"limitador-limitador-54d6d988d5-4xh6w\" (UID: \"ca1f5c56-5803-4889-a36f-4190771c3e63\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.178220 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.178182 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j6hr\" (UniqueName: \"kubernetes.io/projected/ca1f5c56-5803-4889-a36f-4190771c3e63-kube-api-access-5j6hr\") pod \"limitador-limitador-54d6d988d5-4xh6w\" (UID: \"ca1f5c56-5803-4889-a36f-4190771c3e63\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.263602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.263572 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:27.341764 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.341709 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:27.408516 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.408480 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:34:27.411984 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.411954 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:27.413444 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:34:27.413420 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1f5c56_5803_4889_a36f_4190771c3e63.slice/crio-64ff105e315f0585bdc98cabb65d1c25ba7037ac3f029838a3201877d0b894b8 WatchSource:0}: Error finding container 64ff105e315f0585bdc98cabb65d1c25ba7037ac3f029838a3201877d0b894b8: Status 404 returned error can't find the container with id 64ff105e315f0585bdc98cabb65d1c25ba7037ac3f029838a3201877d0b894b8 Jul 14 09:34:27.946218 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:27.946184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" event={"ID":"ca1f5c56-5803-4889-a36f-4190771c3e63","Type":"ContainerStarted","Data":"64ff105e315f0585bdc98cabb65d1c25ba7037ac3f029838a3201877d0b894b8"} Jul 14 09:34:29.776859 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:29.776795 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:29.878105 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:29.878015 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:29.983007 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:29.982971 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:30.084450 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:30.084414 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:30.964838 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:30.964805 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" event={"ID":"ca1f5c56-5803-4889-a36f-4190771c3e63","Type":"ContainerStarted","Data":"635dd1abd3cb48e0114c8445556f29f2d69e303cc7c22523969c2a0ab6b0403c"} Jul 14 09:34:30.965239 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:30.964932 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:30.996475 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:30.996421 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" podStartSLOduration=2.423696667 podStartE2EDuration="4.996405435s" totalImagesPullingTime="2.572708768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:34:30.995316689 +0000 UTC m=+481.933167171" watchObservedRunningTime="2026-07-14 09:34:30.996405435 +0000 UTC m=+481.934255928" Jul 14 09:34:31.569015 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:31.568982 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:31.640794 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:31.640735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:35.051533 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:35.051499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:36.036511 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:36.036474 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:36.133916 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:36.133880 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:41.970697 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:41.970662 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4xh6w" Jul 14 09:34:48.343050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:48.343013 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:34:48.443989 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:34:48.443953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:07.562262 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:07.562225 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:09.382714 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:09.382676 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:09.643289 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:09.643192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:17.483287 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:17.483254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:17.581474 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:17.581443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:20.693581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:20.693485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:20.793219 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:20.793186 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:37.267658 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:37.267571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:37.368076 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:37.368042 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:37.705697 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:37.705617 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:37.974493 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:37.974384 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:38.324541 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:38.324499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:38.396032 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:38.395998 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:35:38.422812 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:38.422775 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:48.970519 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:48.970479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:35:49.065773 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:35:49.065707 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:04.060481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:04.060430 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:04.520850 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:04.520814 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:36:10.953977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:10.953935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:11.719363 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:11.719323 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:36:24.960602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:24.960567 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:27.576599 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:27.576551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:28.462337 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:28.462304 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:29.522519 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:29.522494 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:36:29.524070 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:29.524047 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:36:29.528337 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:29.528309 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:36:37.858020 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:37.857975 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:39.623712 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:39.623677 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:36:41.653529 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:41.651559 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:47.429169 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:47.429132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:48.705521 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:48.705491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:49.416463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:49.416427 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:36:56.665584 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:56.665493 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:36:59.128069 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:36:59.128035 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:37:03.985540 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:37:03.985506 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:37:17.564406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:37:17.564371 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:37:54.120478 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:37:54.120441 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:37:54.175571 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:37:54.175539 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:38:29.080985 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:38:29.080895 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:38:29.178767 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:38:29.178717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:38:30.272796 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:38:30.272759 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:38:30.372825 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:38:30.372796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:38:58.230163 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:38:58.230125 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:39:09.415246 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:39:09.415211 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:39:19.637023 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:39:19.636993 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:39:29.820004 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:39:29.819967 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:39:39.506907 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:39:39.506873 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:39:42.496337 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:39:42.496302 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:39:44.105311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:39:44.105280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:05.923082 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:05.923007 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:40:18.438376 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:18.438337 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:40:23.181853 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:23.181813 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:23.281793 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:23.281737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:29.548631 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:29.548592 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:29.652295 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:29.652266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:29.763252 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:29.763215 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:29.911086 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:29.910989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:34.005769 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:34.005717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:36.377980 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:36.377946 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:36.502478 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:36.502441 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:40:52.328314 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:40:52.328274 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:41:01.626532 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:01.626493 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:41:09.525978 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:09.525941 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:41:16.212493 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:16.212458 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:41:27.021229 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:27.021192 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:41:29.558384 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:29.558353 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:41:29.559320 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:29.559300 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:41:38.527947 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:38.527907 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:41:49.923764 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:41:49.923713 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:42:00.118048 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:42:00.118015 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:42:25.321336 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:42:25.321302 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:42:34.919213 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:42:34.919175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:42:58.635844 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:42:58.635739 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:43:00.742715 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:43:00.742681 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:43:10.119590 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:43:10.119555 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:43:25.119465 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:43:25.119426 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:43:35.917296 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:43:35.917257 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:43:45.729979 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:43:45.729938 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:43:55.412354 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:43:55.412321 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:44:05.211887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:05.211844 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:44:12.935097 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:12.935054 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:44:20.843022 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:20.842987 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:44:29.621449 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:29.621355 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s"] Jul 14 09:44:29.622052 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:29.621640 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" podUID="f0822083-5720-4ec4-9d57-4c1917b8450e" containerName="manager" containerID="cri-o://ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9" gracePeriod=10 Jul 14 09:44:31.588020 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.587996 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:44:31.766968 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.766939 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmsvz\" (UniqueName: \"kubernetes.io/projected/f0822083-5720-4ec4-9d57-4c1917b8450e-kube-api-access-vmsvz\") pod \"f0822083-5720-4ec4-9d57-4c1917b8450e\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " Jul 14 09:44:31.767169 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.767055 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0822083-5720-4ec4-9d57-4c1917b8450e-extensions-socket-volume\") pod \"f0822083-5720-4ec4-9d57-4c1917b8450e\" (UID: \"f0822083-5720-4ec4-9d57-4c1917b8450e\") " Jul 14 09:44:31.767423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.767396 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0822083-5720-4ec4-9d57-4c1917b8450e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f0822083-5720-4ec4-9d57-4c1917b8450e" (UID: "f0822083-5720-4ec4-9d57-4c1917b8450e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:44:31.769132 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.769111 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0822083-5720-4ec4-9d57-4c1917b8450e-kube-api-access-vmsvz" (OuterVolumeSpecName: "kube-api-access-vmsvz") pod "f0822083-5720-4ec4-9d57-4c1917b8450e" (UID: "f0822083-5720-4ec4-9d57-4c1917b8450e"). InnerVolumeSpecName "kube-api-access-vmsvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:44:31.846835 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.846795 2579 generic.go:356] "Generic (PLEG): container finished" podID="f0822083-5720-4ec4-9d57-4c1917b8450e" containerID="ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9" exitCode=0 Jul 14 09:44:31.846835 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.846839 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" Jul 14 09:44:31.847029 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.846875 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" event={"ID":"f0822083-5720-4ec4-9d57-4c1917b8450e","Type":"ContainerDied","Data":"ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9"} Jul 14 09:44:31.847029 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.846932 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s" event={"ID":"f0822083-5720-4ec4-9d57-4c1917b8450e","Type":"ContainerDied","Data":"425df9de305d285d353a178267d6c27a43cf21c8de21bf6eb1bd57e97b388e02"} Jul 14 09:44:31.847029 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.846958 2579 scope.go:117] "RemoveContainer" containerID="ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9" Jul 14 09:44:31.856591 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.856576 2579 scope.go:117] "RemoveContainer" containerID="ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9" Jul 14 09:44:31.856856 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:44:31.856836 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9\": container with ID starting with ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9 not found: ID does not exist" containerID="ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9" Jul 14 09:44:31.856916 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.856863 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9"} err="failed to get container status \"ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9\": rpc error: code = NotFound desc = could not find container \"ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9\": container with ID starting with ae12fd936c5a2772272bc3a5b414e7a093dc0a9a49874998f279ba068c9e28f9 not found: ID does not exist" Jul 14 09:44:31.868097 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.868071 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s"] Jul 14 09:44:31.868381 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.868365 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0822083-5720-4ec4-9d57-4c1917b8450e-extensions-socket-volume\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:44:31.868423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.868384 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmsvz\" (UniqueName: \"kubernetes.io/projected/f0822083-5720-4ec4-9d57-4c1917b8450e-kube-api-access-vmsvz\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:44:31.872185 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:31.872163 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5w6s"] Jul 14 09:44:33.636789 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:44:33.636733 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0822083-5720-4ec4-9d57-4c1917b8450e" path="/var/lib/kubelet/pods/f0822083-5720-4ec4-9d57-4c1917b8450e/volumes" Jul 14 09:45:00.146472 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.146431 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b"] Jul 14 09:45:00.147121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.146976 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0822083-5720-4ec4-9d57-4c1917b8450e" containerName="manager" Jul 14 09:45:00.147121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.146996 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0822083-5720-4ec4-9d57-4c1917b8450e" containerName="manager" Jul 14 09:45:00.147121 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.147118 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0822083-5720-4ec4-9d57-4c1917b8450e" containerName="manager" Jul 14 09:45:00.150151 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.150130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" Jul 14 09:45:00.152427 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.152409 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 09:45:00.156663 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.156639 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b"] Jul 14 09:45:00.161086 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.161059 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" Jul 14 09:45:00.298685 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.298647 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b"] Jul 14 09:45:00.301453 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:45:00.301420 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbb14ca_40af_4357_93e7_82fe51fbfefa.slice/crio-d517e66cdafd8f8ba16c7cb119ec01b2aa7782977643007da982244ea7844522 WatchSource:0}: Error finding container d517e66cdafd8f8ba16c7cb119ec01b2aa7782977643007da982244ea7844522: Status 404 returned error can't find the container with id d517e66cdafd8f8ba16c7cb119ec01b2aa7782977643007da982244ea7844522 Jul 14 09:45:00.303238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.303219 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 09:45:00.980128 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:00.980088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" event={"ID":"bdbb14ca-40af-4357-93e7-82fe51fbfefa","Type":"ContainerStarted","Data":"d517e66cdafd8f8ba16c7cb119ec01b2aa7782977643007da982244ea7844522"} Jul 14 09:45:02.991316 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:02.991274 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" event={"ID":"bdbb14ca-40af-4357-93e7-82fe51fbfefa","Type":"ContainerStarted","Data":"404438a064809835c9157eaf8f3f591aa5d67315ac9f371ddf5b02d1f268b2b6"} Jul 14 09:45:03.011253 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:03.011196 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" podStartSLOduration=0.723464175 podStartE2EDuration="3.011178319s" totalImagesPullingTime="2.287714144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:45:03.009006361 +0000 UTC m=+1113.946856830" watchObservedRunningTime="2026-07-14 09:45:03.011178319 +0000 UTC m=+1113.949028831" Jul 14 09:45:03.997566 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:03.997530 2579 generic.go:356] "Generic (PLEG): container finished" podID="bdbb14ca-40af-4357-93e7-82fe51fbfefa" containerID="404438a064809835c9157eaf8f3f591aa5d67315ac9f371ddf5b02d1f268b2b6" exitCode=0 Jul 14 09:45:03.997997 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:03.997596 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" event={"ID":"bdbb14ca-40af-4357-93e7-82fe51fbfefa","Type":"ContainerDied","Data":"404438a064809835c9157eaf8f3f591aa5d67315ac9f371ddf5b02d1f268b2b6"} Jul 14 09:45:05.141150 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:05.141125 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" Jul 14 09:45:06.008596 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:06.008569 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" Jul 14 09:45:06.008851 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:06.008598 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733705-rdb9b" event={"ID":"bdbb14ca-40af-4357-93e7-82fe51fbfefa","Type":"ContainerDied","Data":"d517e66cdafd8f8ba16c7cb119ec01b2aa7782977643007da982244ea7844522"} Jul 14 09:45:06.008851 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:06.008640 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d517e66cdafd8f8ba16c7cb119ec01b2aa7782977643007da982244ea7844522" Jul 14 09:45:30.787489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.787454 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p"] Jul 14 09:45:30.787924 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.787818 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdbb14ca-40af-4357-93e7-82fe51fbfefa" containerName="cleanup" Jul 14 09:45:30.787924 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.787829 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbb14ca-40af-4357-93e7-82fe51fbfefa" containerName="cleanup" Jul 14 09:45:30.787924 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.787894 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdbb14ca-40af-4357-93e7-82fe51fbfefa" containerName="cleanup" Jul 14 09:45:30.791181 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.791145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:30.797794 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.797774 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l4nht\"" Jul 14 09:45:30.811223 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.811198 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p"] Jul 14 09:45:30.983439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.983399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df00865d-b84e-4917-9ca9-e4846c081e1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hzq2p\" (UID: \"df00865d-b84e-4917-9ca9-e4846c081e1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:30.983439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:30.983446 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bslbm\" (UniqueName: \"kubernetes.io/projected/df00865d-b84e-4917-9ca9-e4846c081e1f-kube-api-access-bslbm\") pod \"kuadrant-operator-controller-manager-86878c769f-hzq2p\" (UID: \"df00865d-b84e-4917-9ca9-e4846c081e1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:31.084479 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:31.084385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df00865d-b84e-4917-9ca9-e4846c081e1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hzq2p\" (UID: \"df00865d-b84e-4917-9ca9-e4846c081e1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:31.084479 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:31.084431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bslbm\" (UniqueName: \"kubernetes.io/projected/df00865d-b84e-4917-9ca9-e4846c081e1f-kube-api-access-bslbm\") pod \"kuadrant-operator-controller-manager-86878c769f-hzq2p\" (UID: \"df00865d-b84e-4917-9ca9-e4846c081e1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:31.084814 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:31.084795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df00865d-b84e-4917-9ca9-e4846c081e1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hzq2p\" (UID: \"df00865d-b84e-4917-9ca9-e4846c081e1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:31.096455 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:31.096429 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bslbm\" (UniqueName: \"kubernetes.io/projected/df00865d-b84e-4917-9ca9-e4846c081e1f-kube-api-access-bslbm\") pod \"kuadrant-operator-controller-manager-86878c769f-hzq2p\" (UID: \"df00865d-b84e-4917-9ca9-e4846c081e1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:31.101303 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:31.101274 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:31.253950 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:31.253922 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p"] Jul 14 09:45:31.257077 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:45:31.257055 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf00865d_b84e_4917_9ca9_e4846c081e1f.slice/crio-5bfdc910279a6670edfa28662506ec6b91a3c51975be7dd83d39ed1255320597 WatchSource:0}: Error finding container 5bfdc910279a6670edfa28662506ec6b91a3c51975be7dd83d39ed1255320597: Status 404 returned error can't find the container with id 5bfdc910279a6670edfa28662506ec6b91a3c51975be7dd83d39ed1255320597 Jul 14 09:45:32.150369 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:32.150333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" event={"ID":"df00865d-b84e-4917-9ca9-e4846c081e1f","Type":"ContainerStarted","Data":"b77407e9e5f79294f7c92032bf7777717dd9b9ca132b2e12694ab1acc65a1b01"} Jul 14 09:45:32.150840 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:32.150391 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" event={"ID":"df00865d-b84e-4917-9ca9-e4846c081e1f","Type":"ContainerStarted","Data":"5bfdc910279a6670edfa28662506ec6b91a3c51975be7dd83d39ed1255320597"} Jul 14 09:45:32.150840 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:32.150499 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:32.175566 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:32.175506 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" podStartSLOduration=2.175459019 podStartE2EDuration="2.175459019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:45:32.172168122 +0000 UTC m=+1143.110018591" watchObservedRunningTime="2026-07-14 09:45:32.175459019 +0000 UTC m=+1143.113309491" Jul 14 09:45:43.157872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:43.157842 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hzq2p" Jul 14 09:45:51.161384 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:45:51.161349 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:46:02.732655 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:02.732623 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:46:12.134594 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:12.134558 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:46:23.492335 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:23.492302 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:46:23.581672 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:23.581641 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:46:26.322547 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:26.322507 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:46:29.587909 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:29.587859 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:46:29.589889 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:29.589864 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:46:36.022397 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:36.022361 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:46:45.222597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:45.222555 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:46:55.522196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:46:55.522159 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:47:05.214311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:47:05.214272 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:47:16.519439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:47:16.519408 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:47:24.154945 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:47:24.154855 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:47:33.222670 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:47:33.222634 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:47:41.325515 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:47:41.325479 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:48:00.517719 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:48:00.517681 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:48:09.732239 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:48:09.732195 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:48:19.321597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:48:19.321564 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:48:30.427227 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:48:30.427185 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:48:38.319281 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:48:38.319242 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:48:51.319965 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:48:51.319933 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:49:00.619567 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:49:00.619529 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:49:09.715568 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:49:09.715518 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:49:18.820332 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:49:18.820293 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:49:28.020846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:49:28.020809 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:49:37.317405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:49:37.317364 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:49:50.845850 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:49:50.845815 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:50:00.129899 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:50:00.129864 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:50:09.615843 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:50:09.615807 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:50:20.219172 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:50:20.219123 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:50:31.314124 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:50:31.314041 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:50:43.786713 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:50:43.786674 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:50:53.548828 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:50:53.548790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:51:02.892059 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:02.892021 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:51:09.621667 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:09.621629 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:51:24.727024 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:24.726992 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:51:25.001387 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:25.001281 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:51:29.614846 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:29.614812 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:51:29.619456 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:29.619436 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:51:34.731394 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:34.731359 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:51:38.708694 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:51:38.708663 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:02.562405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:02.562300 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:05.523720 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:05.523675 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:52:10.771401 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:10.771369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:15.937190 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:15.937158 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:18.957178 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:18.957144 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:52:19.250872 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:19.250788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:19.363939 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:19.363909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:19.806584 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:19.806555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:25.752029 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:25.751997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:25.852627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:25.852571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:25.948813 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:25.948781 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p"] Jul 14 09:52:25.952439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:25.952420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:25.955717 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:25.955694 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c9216ec8-openshift-default-dockercfg-mddsn\"" Jul 14 09:52:25.992406 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:25.992373 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p"] Jul 14 09:52:26.003664 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.003569 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:26.089210 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.089183 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:26.096009 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.095984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3893a2bc-5d61-435b-9275-21d3d56d6a54-istiod-ca-cert\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096131 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096017 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-data\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096131 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096046 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-podinfo\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096131 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-token\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-socket\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8p5d\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-kube-api-access-m8p5d\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096215 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-envoy\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096344 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-credential-socket\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.096344 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.096274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-certs\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197528 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-podinfo\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197528 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-token\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-socket\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197582 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8p5d\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-kube-api-access-m8p5d\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-envoy\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-credential-socket\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.197809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-certs\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198064 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3893a2bc-5d61-435b-9275-21d3d56d6a54-istiod-ca-cert\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198064 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.197892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-data\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198165 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.198062 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-certs\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198165 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.198098 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-socket\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198319 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.198292 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-data\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.198391 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-credential-socket\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.198522 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.198457 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3893a2bc-5d61-435b-9275-21d3d56d6a54-istiod-ca-cert\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.199962 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.199942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-envoy\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.200151 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.200010 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-podinfo\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.208719 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.208699 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8p5d\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-kube-api-access-m8p5d\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.208853 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.208837 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-token\") pod \"e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.265021 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.264985 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:26.411181 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.410575 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p"] Jul 14 09:52:26.414026 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.413990 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:26.414112 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.414061 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:26.414112 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:26.414090 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:27.169020 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.168979 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" event={"ID":"3893a2bc-5d61-435b-9275-21d3d56d6a54","Type":"ContainerStarted","Data":"c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20"} Jul 14 09:52:27.169413 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.169051 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" event={"ID":"3893a2bc-5d61-435b-9275-21d3d56d6a54","Type":"ContainerStarted","Data":"527450b31f1de24f352dd353d27e26f53b17efa5ae40c6f9c6b002443f75aba7"} Jul 14 09:52:27.192190 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.192155 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:27.202380 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.202339 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podStartSLOduration=2.202323995 podStartE2EDuration="2.202323995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:52:27.200502476 +0000 UTC m=+1558.138352967" watchObservedRunningTime="2026-07-14 09:52:27.202323995 +0000 UTC m=+1558.140174463" Jul 14 09:52:27.265120 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.265089 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:27.270362 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.270332 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:27.294950 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.294920 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:27.480210 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.480130 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:27.600423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.600394 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:27.972419 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:27.972386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:28.070375 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:28.070340 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:28.174119 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:28.174088 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:28.174926 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:28.174909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:29.904935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:29.904907 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:32.130881 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.130848 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:52:32.207898 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.207867 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw"] Jul 14 09:52:32.211506 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.211483 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:32.214211 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.214182 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 09:52:32.214324 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.214214 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 09:52:32.220335 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.220314 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw"] Jul 14 09:52:32.354048 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.354007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/794bb54e-183b-432b-acaa-b346e9581a50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw\" (UID: \"794bb54e-183b-432b-acaa-b346e9581a50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:32.454968 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.454875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/794bb54e-183b-432b-acaa-b346e9581a50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw\" (UID: \"794bb54e-183b-432b-acaa-b346e9581a50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:32.455540 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.455519 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/794bb54e-183b-432b-acaa-b346e9581a50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw\" (UID: \"794bb54e-183b-432b-acaa-b346e9581a50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:32.522145 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.522089 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:32.658826 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:32.658797 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw"] Jul 14 09:52:32.661704 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:52:32.661665 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794bb54e_183b_432b_acaa_b346e9581a50.slice/crio-90d820003f1579b0534ff0a5dac23348bb493aee3402b97689479e8d2492191a WatchSource:0}: Error finding container 90d820003f1579b0534ff0a5dac23348bb493aee3402b97689479e8d2492191a: Status 404 returned error can't find the container with id 90d820003f1579b0534ff0a5dac23348bb493aee3402b97689479e8d2492191a Jul 14 09:52:33.198368 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:33.198282 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" event={"ID":"794bb54e-183b-432b-acaa-b346e9581a50","Type":"ContainerStarted","Data":"54e82f9f856e74690e6bf94e9227823261e472fb185a9f94b2a35a669aba0e37"} Jul 14 09:52:33.198368 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:33.198335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" event={"ID":"794bb54e-183b-432b-acaa-b346e9581a50","Type":"ContainerStarted","Data":"90d820003f1579b0534ff0a5dac23348bb493aee3402b97689479e8d2492191a"} Jul 14 09:52:33.217378 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:33.217320 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" podStartSLOduration=1.217301146 podStartE2EDuration="1.217301146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:52:33.216792046 +0000 UTC m=+1564.154642515" watchObservedRunningTime="2026-07-14 09:52:33.217301146 +0000 UTC m=+1564.155151615" Jul 14 09:52:34.205248 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:34.205217 2579 generic.go:356] "Generic (PLEG): container finished" podID="794bb54e-183b-432b-acaa-b346e9581a50" containerID="54e82f9f856e74690e6bf94e9227823261e472fb185a9f94b2a35a669aba0e37" exitCode=7 Jul 14 09:52:34.205836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:34.205269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" event={"ID":"794bb54e-183b-432b-acaa-b346e9581a50","Type":"ContainerDied","Data":"54e82f9f856e74690e6bf94e9227823261e472fb185a9f94b2a35a669aba0e37"} Jul 14 09:52:34.205836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:34.205522 2579 scope.go:117] "RemoveContainer" containerID="54e82f9f856e74690e6bf94e9227823261e472fb185a9f94b2a35a669aba0e37" Jul 14 09:52:35.211971 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:35.211941 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" event={"ID":"794bb54e-183b-432b-acaa-b346e9581a50","Type":"ContainerStarted","Data":"8edd70abd88b66f58b6d1905e629b8ab88d8ecc8130b1a8376b2da0ac041a144"} Jul 14 09:52:36.218847 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:36.218761 2579 generic.go:356] "Generic (PLEG): container finished" podID="794bb54e-183b-432b-acaa-b346e9581a50" containerID="8edd70abd88b66f58b6d1905e629b8ab88d8ecc8130b1a8376b2da0ac041a144" exitCode=0 Jul 14 09:52:36.218847 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:36.218797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" event={"ID":"794bb54e-183b-432b-acaa-b346e9581a50","Type":"ContainerDied","Data":"8edd70abd88b66f58b6d1905e629b8ab88d8ecc8130b1a8376b2da0ac041a144"} Jul 14 09:52:36.218847 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:36.218843 2579 scope.go:117] "RemoveContainer" containerID="54e82f9f856e74690e6bf94e9227823261e472fb185a9f94b2a35a669aba0e37" Jul 14 09:52:37.407217 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:37.407192 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:37.508179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:37.508155 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/794bb54e-183b-432b-acaa-b346e9581a50-maas-api-service-ca\") pod \"794bb54e-183b-432b-acaa-b346e9581a50\" (UID: \"794bb54e-183b-432b-acaa-b346e9581a50\") " Jul 14 09:52:37.508517 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:37.508494 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/794bb54e-183b-432b-acaa-b346e9581a50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "794bb54e-183b-432b-acaa-b346e9581a50" (UID: "794bb54e-183b-432b-acaa-b346e9581a50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:52:37.609282 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:37.609241 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/794bb54e-183b-432b-acaa-b346e9581a50-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:52:38.231414 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:38.231382 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" Jul 14 09:52:38.231607 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:38.231381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw" event={"ID":"794bb54e-183b-432b-acaa-b346e9581a50","Type":"ContainerDied","Data":"90d820003f1579b0534ff0a5dac23348bb493aee3402b97689479e8d2492191a"} Jul 14 09:52:38.231607 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:38.231501 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90d820003f1579b0534ff0a5dac23348bb493aee3402b97689479e8d2492191a" Jul 14 09:52:41.246931 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:41.246895 2579 generic.go:356] "Generic (PLEG): container finished" podID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerID="c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20" exitCode=0 Jul 14 09:52:41.247355 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:41.246967 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" event={"ID":"3893a2bc-5d61-435b-9275-21d3d56d6a54","Type":"ContainerDied","Data":"c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20"} Jul 14 09:52:41.247355 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:41.247340 2579 scope.go:117] "RemoveContainer" containerID="c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20" Jul 14 09:52:41.248296 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:41.248268 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:41.248363 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:41.248336 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:41.248400 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:41.248384 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:42.253196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:42.253161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" event={"ID":"3893a2bc-5d61-435b-9275-21d3d56d6a54","Type":"ContainerStarted","Data":"a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4"} Jul 14 09:52:42.266213 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:42.266162 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:42.267612 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:42.267573 2579 patch_prober.go:28] interesting pod/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 14 09:52:42.267785 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:42.267657 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 14 09:52:42.696375 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:42.696330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:43.174056 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:43.174014 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:43.265839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:43.265801 2579 patch_prober.go:28] interesting pod/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 14 09:52:43.266310 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:43.265874 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 14 09:52:44.143063 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:44.143032 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:44.268809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:44.268771 2579 patch_prober.go:28] interesting pod/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 14 09:52:44.269411 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:44.269375 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 14 09:52:45.265678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:45.265639 2579 patch_prober.go:28] interesting pod/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 14 09:52:45.266003 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:45.265972 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 14 09:52:46.270154 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:46.270122 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:46.271266 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:46.271245 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:52:46.824874 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:46.824837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:52:47.311481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:47.311451 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:53.677128 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:53.677093 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:53.777119 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:53.777082 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:57.476140 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:57.476099 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p"] Jul 14 09:52:57.476628 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:57.476366 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" containerID="cri-o://a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4" gracePeriod=30 Jul 14 09:52:57.495144 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:57.495114 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:57.862903 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:57.862868 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:57.962860 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:57.962811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:58.018589 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.018553 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8"] Jul 14 09:52:58.019135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.019120 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794bb54e-183b-432b-acaa-b346e9581a50" containerName="revoke-keys" Jul 14 09:52:58.019199 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.019139 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="794bb54e-183b-432b-acaa-b346e9581a50" containerName="revoke-keys" Jul 14 09:52:58.019199 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.019162 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794bb54e-183b-432b-acaa-b346e9581a50" containerName="revoke-keys" Jul 14 09:52:58.019199 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.019171 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="794bb54e-183b-432b-acaa-b346e9581a50" containerName="revoke-keys" Jul 14 09:52:58.019289 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.019271 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="794bb54e-183b-432b-acaa-b346e9581a50" containerName="revoke-keys" Jul 14 09:52:58.019289 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.019284 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="794bb54e-183b-432b-acaa-b346e9581a50" containerName="revoke-keys" Jul 14 09:52:58.023259 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.023238 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.025923 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.025901 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-40d96159-gw-openshift-default-dockercfg-6qn2q\"" Jul 14 09:52:58.039246 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.039220 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8"] Jul 14 09:52:58.083785 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.083732 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:58.104076 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-token\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104076 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104074 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjfh8\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-kube-api-access-gjfh8\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104321 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-credential-socket\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104321 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-data\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104321 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8438bd75-3a35-426f-9984-cfb6e2ff345f-istiod-ca-cert\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104320 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-podinfo\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-certs\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-envoy\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.104481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.104462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-socket\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205208 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-socket\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205208 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-token\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205208 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205198 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjfh8\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-kube-api-access-gjfh8\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205208 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-credential-socket\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205500 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-data\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205500 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8438bd75-3a35-426f-9984-cfb6e2ff345f-istiod-ca-cert\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205500 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-podinfo\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205652 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205497 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-socket\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205652 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-certs\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205652 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-envoy\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205848 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205711 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-credential-socket\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205908 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205850 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-data\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.205983 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.205960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-certs\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.206078 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.206059 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8438bd75-3a35-426f-9984-cfb6e2ff345f-istiod-ca-cert\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.207775 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.207735 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-envoy\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.207992 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.207975 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-podinfo\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.213498 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.213477 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-token\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.214481 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.214460 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjfh8\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-kube-api-access-gjfh8\") pod \"e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.333761 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.333718 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:52:58.481705 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.481615 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8"] Jul 14 09:52:58.484626 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:52:58.484583 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8438bd75_3a35_426f_9984_cfb6e2ff345f.slice/crio-7242b501ffa2b1017fbc8a1a766f1e48f19ce74e6084aab0d0587784c0ac1cea WatchSource:0}: Error finding container 7242b501ffa2b1017fbc8a1a766f1e48f19ce74e6084aab0d0587784c0ac1cea: Status 404 returned error can't find the container with id 7242b501ffa2b1017fbc8a1a766f1e48f19ce74e6084aab0d0587784c0ac1cea Jul 14 09:52:58.487066 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.487033 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:58.487137 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.487096 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:58.487137 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:58.487121 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:52:59.276489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:59.276461 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:59.349190 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:59.349157 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" event={"ID":"8438bd75-3a35-426f-9984-cfb6e2ff345f","Type":"ContainerStarted","Data":"7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2"} Jul 14 09:52:59.349391 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:59.349203 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" event={"ID":"8438bd75-3a35-426f-9984-cfb6e2ff345f","Type":"ContainerStarted","Data":"7242b501ffa2b1017fbc8a1a766f1e48f19ce74e6084aab0d0587784c0ac1cea"} Jul 14 09:52:59.374225 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:59.374178 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" podStartSLOduration=2.374163148 podStartE2EDuration="2.374163148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:52:59.371460158 +0000 UTC m=+1590.309310631" watchObservedRunningTime="2026-07-14 09:52:59.374163148 +0000 UTC m=+1590.312013615" Jul 14 09:52:59.517587 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:59.517554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:52:59.709022 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:52:59.708929 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:53:00.165244 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:00.165200 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:53:00.267871 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:00.267836 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:53:00.334129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:00.334094 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:00.338960 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:00.338934 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:00.354260 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:00.354223 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:00.355139 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:00.355121 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:01.272780 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:01.272715 2579 patch_prober.go:28] interesting pod/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 09:53:01.273243 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:01.272817 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 09:53:02.736837 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.736811 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:53:02.848453 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848423 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-token\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848570 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848489 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-podinfo\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848570 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848546 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-socket\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848569 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-envoy\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848594 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-credential-socket\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848638 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3893a2bc-5d61-435b-9275-21d3d56d6a54-istiod-ca-cert\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848712 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-data\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848780 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-certs\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848836 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848824 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8p5d\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-kube-api-access-m8p5d\") pod \"3893a2bc-5d61-435b-9275-21d3d56d6a54\" (UID: \"3893a2bc-5d61-435b-9275-21d3d56d6a54\") " Jul 14 09:53:02.848994 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848855 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:53:02.848994 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.848963 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:53:02.849388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.849164 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.849388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.849189 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.849388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.849251 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3893a2bc-5d61-435b-9275-21d3d56d6a54-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:53:02.849388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.849354 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:53:02.849696 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.849661 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-data" (OuterVolumeSpecName: "istio-data") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:53:02.850995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.850953 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:53:02.851189 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.851145 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 09:53:02.851282 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.851245 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-token" (OuterVolumeSpecName: "istio-token") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:53:02.851459 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.851437 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-kube-api-access-m8p5d" (OuterVolumeSpecName: "kube-api-access-m8p5d") pod "3893a2bc-5d61-435b-9275-21d3d56d6a54" (UID: "3893a2bc-5d61-435b-9275-21d3d56d6a54"). InnerVolumeSpecName "kube-api-access-m8p5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:53:02.949626 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949592 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3893a2bc-5d61-435b-9275-21d3d56d6a54-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.949626 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949618 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.949626 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949627 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.949626 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949635 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8p5d\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-kube-api-access-m8p5d\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.949935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949644 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.949935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949652 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:02.949935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:02.949661 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3893a2bc-5d61-435b-9275-21d3d56d6a54-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:03.370572 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.370539 2579 generic.go:356] "Generic (PLEG): container finished" podID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerID="a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4" exitCode=0 Jul 14 09:53:03.370758 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.370580 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" Jul 14 09:53:03.370758 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.370589 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" event={"ID":"3893a2bc-5d61-435b-9275-21d3d56d6a54","Type":"ContainerDied","Data":"a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4"} Jul 14 09:53:03.370758 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.370646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p" event={"ID":"3893a2bc-5d61-435b-9275-21d3d56d6a54","Type":"ContainerDied","Data":"527450b31f1de24f352dd353d27e26f53b17efa5ae40c6f9c6b002443f75aba7"} Jul 14 09:53:03.370758 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.370672 2579 scope.go:117] "RemoveContainer" containerID="a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4" Jul 14 09:53:03.380290 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.380275 2579 scope.go:117] "RemoveContainer" containerID="c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20" Jul 14 09:53:03.388168 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.388151 2579 scope.go:117] "RemoveContainer" containerID="a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4" Jul 14 09:53:03.388439 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:53:03.388421 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4\": container with ID starting with a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4 not found: ID does not exist" containerID="a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4" Jul 14 09:53:03.388489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.388449 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4"} err="failed to get container status \"a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4\": rpc error: code = NotFound desc = could not find container \"a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4\": container with ID starting with a354d52241ca1dda7b64756ccbb1880e0f57be691578f5c1502cf3d7f002efb4 not found: ID does not exist" Jul 14 09:53:03.388489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.388466 2579 scope.go:117] "RemoveContainer" containerID="c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20" Jul 14 09:53:03.388708 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:53:03.388687 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20\": container with ID starting with c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20 not found: ID does not exist" containerID="c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20" Jul 14 09:53:03.388766 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.388715 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20"} err="failed to get container status \"c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20\": rpc error: code = NotFound desc = could not find container \"c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20\": container with ID starting with c59b8332742a1295656dec55ba79365fafa3142b437f029bed5a4d14398dac20 not found: ID does not exist" Jul 14 09:53:03.397939 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.397909 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p"] Jul 14 09:53:03.405410 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.405384 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c9216ec8-openshift-default-5dcc7cfd9c-pp58p"] Jul 14 09:53:03.637425 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.637341 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" path="/var/lib/kubelet/pods/3893a2bc-5d61-435b-9275-21d3d56d6a54/volumes" Jul 14 09:53:03.799230 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.799192 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7"] Jul 14 09:53:03.799598 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.799560 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" Jul 14 09:53:03.799598 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.799570 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" Jul 14 09:53:03.799672 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.799642 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" Jul 14 09:53:03.799672 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.799651 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" Jul 14 09:53:03.804283 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.804263 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:03.806729 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.806711 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 09:53:03.806855 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.806796 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 09:53:03.812946 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.812922 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7"] Jul 14 09:53:03.860157 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.860108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40d96159-698j7\" (UID: \"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:03.961135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.961032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40d96159-698j7\" (UID: \"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:03.961703 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:03.961677 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40d96159-698j7\" (UID: \"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:04.115174 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:04.115130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:04.259146 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:04.259114 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7"] Jul 14 09:53:04.262351 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:53:04.262319 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b3b59a_fd16_4e39_b4fa_d54c15d72d90.slice/crio-1c791f577416adacc1f71717bea2f837e887e592185f8c2506ba10ff24f92b29 WatchSource:0}: Error finding container 1c791f577416adacc1f71717bea2f837e887e592185f8c2506ba10ff24f92b29: Status 404 returned error can't find the container with id 1c791f577416adacc1f71717bea2f837e887e592185f8c2506ba10ff24f92b29 Jul 14 09:53:04.377299 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:04.377262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" event={"ID":"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90","Type":"ContainerStarted","Data":"1c791f577416adacc1f71717bea2f837e887e592185f8c2506ba10ff24f92b29"} Jul 14 09:53:04.522861 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:04.522780 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:53:05.385648 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:05.385616 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerID="63366cf9836c1144bc175bfe05798c69ecf68145c00143cb72a7be3197268300" exitCode=7 Jul 14 09:53:05.386124 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:05.385681 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" event={"ID":"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90","Type":"ContainerDied","Data":"63366cf9836c1144bc175bfe05798c69ecf68145c00143cb72a7be3197268300"} Jul 14 09:53:05.386124 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:05.385876 2579 scope.go:117] "RemoveContainer" containerID="63366cf9836c1144bc175bfe05798c69ecf68145c00143cb72a7be3197268300" Jul 14 09:53:06.393473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:06.393441 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerID="62387fc543c95f0d641a953addad9082f25a280fd69c3dcde38701376e971300" exitCode=7 Jul 14 09:53:06.393893 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:06.393528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" event={"ID":"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90","Type":"ContainerDied","Data":"62387fc543c95f0d641a953addad9082f25a280fd69c3dcde38701376e971300"} Jul 14 09:53:06.393893 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:06.393583 2579 scope.go:117] "RemoveContainer" containerID="63366cf9836c1144bc175bfe05798c69ecf68145c00143cb72a7be3197268300" Jul 14 09:53:06.393893 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:06.393729 2579 scope.go:117] "RemoveContainer" containerID="62387fc543c95f0d641a953addad9082f25a280fd69c3dcde38701376e971300" Jul 14 09:53:06.394048 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:53:06.393976 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40d96159-698j7_odh-ai-gateway-infra(e6b3b59a-fd16-4e39-b4fa-d54c15d72d90)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" Jul 14 09:53:07.401230 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:07.401203 2579 scope.go:117] "RemoveContainer" containerID="62387fc543c95f0d641a953addad9082f25a280fd69c3dcde38701376e971300" Jul 14 09:53:07.401638 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:53:07.401431 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-40d96159-698j7_odh-ai-gateway-infra(e6b3b59a-fd16-4e39-b4fa-d54c15d72d90)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" Jul 14 09:53:08.803421 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:08.803385 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:53:08.905535 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:08.905501 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:53:12.202865 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:12.202833 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:53:13.432814 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:13.432776 2579 generic.go:356] "Generic (PLEG): container finished" podID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerID="7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2" exitCode=0 Jul 14 09:53:13.433234 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:13.432788 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" event={"ID":"8438bd75-3a35-426f-9984-cfb6e2ff345f","Type":"ContainerDied","Data":"7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2"} Jul 14 09:53:13.433277 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:13.433258 2579 scope.go:117] "RemoveContainer" containerID="7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2" Jul 14 09:53:13.434310 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:13.434279 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:53:13.434383 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:13.434366 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:53:13.434433 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:13.434419 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:53:14.439649 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:14.439615 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" event={"ID":"8438bd75-3a35-426f-9984-cfb6e2ff345f","Type":"ContainerStarted","Data":"6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88"} Jul 14 09:53:15.334608 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:15.334566 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:15.335820 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:15.335793 2579 patch_prober.go:28] interesting pod/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 14 09:53:15.335942 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:15.335841 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 14 09:53:15.354606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:15.354578 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:16.334526 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:16.334485 2579 patch_prober.go:28] interesting pod/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 14 09:53:16.334951 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:16.334557 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 14 09:53:17.334856 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:17.334815 2579 patch_prober.go:28] interesting pod/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 14 09:53:17.335269 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:17.334883 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 14 09:53:18.338457 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:18.338429 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:18.339358 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:18.339337 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:53:19.633489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:19.633458 2579 scope.go:117] "RemoveContainer" containerID="62387fc543c95f0d641a953addad9082f25a280fd69c3dcde38701376e971300" Jul 14 09:53:20.474419 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:20.474387 2579 generic.go:356] "Generic (PLEG): container finished" podID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerID="c6b248cbdc184ae423850bbb3b475839eb9afe0ffc560c28ddafba8d5e927a46" exitCode=0 Jul 14 09:53:20.474591 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:20.474447 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" event={"ID":"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90","Type":"ContainerDied","Data":"c6b248cbdc184ae423850bbb3b475839eb9afe0ffc560c28ddafba8d5e927a46"} Jul 14 09:53:20.474591 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:20.474517 2579 scope.go:117] "RemoveContainer" containerID="62387fc543c95f0d641a953addad9082f25a280fd69c3dcde38701376e971300" Jul 14 09:53:21.497909 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:21.497877 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7"] Jul 14 09:53:21.617901 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:21.617875 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:21.735543 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:21.735443 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90-maas-api-service-ca\") pod \"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90\" (UID: \"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90\") " Jul 14 09:53:21.735862 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:21.735825 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" (UID: "e6b3b59a-fd16-4e39-b4fa-d54c15d72d90"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:53:21.836463 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:21.836422 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:22.487231 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:22.487198 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" Jul 14 09:53:22.487231 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:22.487212 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7" event={"ID":"e6b3b59a-fd16-4e39-b4fa-d54c15d72d90","Type":"ContainerDied","Data":"1c791f577416adacc1f71717bea2f837e887e592185f8c2506ba10ff24f92b29"} Jul 14 09:53:22.487455 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:22.487263 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c791f577416adacc1f71717bea2f837e887e592185f8c2506ba10ff24f92b29" Jul 14 09:53:22.526030 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:22.525997 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7"] Jul 14 09:53:22.532605 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:22.532580 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-698j7"] Jul 14 09:53:23.637069 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:23.637034 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" path="/var/lib/kubelet/pods/e6b3b59a-fd16-4e39-b4fa-d54c15d72d90/volumes" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.853322 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47"] Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854228 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854247 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854305 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854314 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3893a2bc-5d61-435b-9275-21d3d56d6a54" containerName="istio-proxy" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854332 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854341 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854366 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.854482 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854374 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.855620 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854536 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.855620 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.854558 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:53:53.859650 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.859619 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:53.862570 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.862547 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 09:53:53.863077 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.863060 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 09:53:53.866678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.866653 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47"] Jul 14 09:53:53.914434 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:53.914395 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b1877-827f-438d-902e-02f58a560d29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40d96159-lwh47\" (UID: \"a57b1877-827f-438d-902e-02f58a560d29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:54.015887 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.015849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b1877-827f-438d-902e-02f58a560d29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40d96159-lwh47\" (UID: \"a57b1877-827f-438d-902e-02f58a560d29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:54.016450 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.016430 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b1877-827f-438d-902e-02f58a560d29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-40d96159-lwh47\" (UID: \"a57b1877-827f-438d-902e-02f58a560d29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:54.172547 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.172455 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:54.313301 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.313279 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47"] Jul 14 09:53:54.315878 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:53:54.315841 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57b1877_827f_438d_902e_02f58a560d29.slice/crio-6abbfa10d32b95e003eda84970a7fdf33856301d29009644091dc187798e3086 WatchSource:0}: Error finding container 6abbfa10d32b95e003eda84970a7fdf33856301d29009644091dc187798e3086: Status 404 returned error can't find the container with id 6abbfa10d32b95e003eda84970a7fdf33856301d29009644091dc187798e3086 Jul 14 09:53:54.646311 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.646277 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" event={"ID":"a57b1877-827f-438d-902e-02f58a560d29","Type":"ContainerStarted","Data":"95695881ae4d735fd179a3218417d1d209ae4c828ab9c228f589f4beb5647be7"} Jul 14 09:53:54.646490 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.646325 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" event={"ID":"a57b1877-827f-438d-902e-02f58a560d29","Type":"ContainerStarted","Data":"6abbfa10d32b95e003eda84970a7fdf33856301d29009644091dc187798e3086"} Jul 14 09:53:54.672101 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:54.672047 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" podStartSLOduration=1.672032376 podStartE2EDuration="1.672032376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:53:54.670605318 +0000 UTC m=+1645.608455789" watchObservedRunningTime="2026-07-14 09:53:54.672032376 +0000 UTC m=+1645.609882846" Jul 14 09:53:55.652178 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:55.652146 2579 generic.go:356] "Generic (PLEG): container finished" podID="a57b1877-827f-438d-902e-02f58a560d29" containerID="95695881ae4d735fd179a3218417d1d209ae4c828ab9c228f589f4beb5647be7" exitCode=0 Jul 14 09:53:55.652559 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:55.652229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" event={"ID":"a57b1877-827f-438d-902e-02f58a560d29","Type":"ContainerDied","Data":"95695881ae4d735fd179a3218417d1d209ae4c828ab9c228f589f4beb5647be7"} Jul 14 09:53:56.791949 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:56.791924 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:56.841669 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:56.841638 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b1877-827f-438d-902e-02f58a560d29-maas-api-service-ca\") pod \"a57b1877-827f-438d-902e-02f58a560d29\" (UID: \"a57b1877-827f-438d-902e-02f58a560d29\") " Jul 14 09:53:56.841962 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:56.841941 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a57b1877-827f-438d-902e-02f58a560d29-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a57b1877-827f-438d-902e-02f58a560d29" (UID: "a57b1877-827f-438d-902e-02f58a560d29"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:53:56.942671 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:56.942585 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b1877-827f-438d-902e-02f58a560d29-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:53:57.662957 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:57.662929 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" Jul 14 09:53:57.662957 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:57.662936 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-40d96159-lwh47" event={"ID":"a57b1877-827f-438d-902e-02f58a560d29","Type":"ContainerDied","Data":"6abbfa10d32b95e003eda84970a7fdf33856301d29009644091dc187798e3086"} Jul 14 09:53:57.663139 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:53:57.662980 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6abbfa10d32b95e003eda84970a7fdf33856301d29009644091dc187798e3086" Jul 14 09:54:03.897518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:03.897484 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8"] Jul 14 09:54:03.898052 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:03.897705 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" containerID="cri-o://6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88" gracePeriod=30 Jul 14 09:54:03.928203 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:03.928173 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:04.053863 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:04.053831 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:04.298225 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:04.298190 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:06.720146 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:06.720110 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:54:09.165363 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.165295 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:54:09.257586 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257559 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-podinfo\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257729 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257596 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-certs\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257729 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257636 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjfh8\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-kube-api-access-gjfh8\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257869 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257763 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-envoy\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257869 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257825 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-credential-socket\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257981 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257875 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-data\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257981 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257908 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-socket\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.257981 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257922 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:09.257981 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257957 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-token\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.258189 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.257984 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8438bd75-3a35-426f-9984-cfb6e2ff345f-istiod-ca-cert\") pod \"8438bd75-3a35-426f-9984-cfb6e2ff345f\" (UID: \"8438bd75-3a35-426f-9984-cfb6e2ff345f\") " Jul 14 09:54:09.258548 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.258280 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.258548 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.258295 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:09.258548 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.258307 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:09.258766 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.258633 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8438bd75-3a35-426f-9984-cfb6e2ff345f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:54:09.258766 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.258692 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:09.260423 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.260356 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-kube-api-access-gjfh8" (OuterVolumeSpecName: "kube-api-access-gjfh8") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "kube-api-access-gjfh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:54:09.260525 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.260469 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:09.260983 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.260945 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:54:09.260983 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.260968 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8438bd75-3a35-426f-9984-cfb6e2ff345f" (UID: "8438bd75-3a35-426f-9984-cfb6e2ff345f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 09:54:09.359597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359558 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359591 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359600 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359608 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359617 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8438bd75-3a35-426f-9984-cfb6e2ff345f-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359625 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359633 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjfh8\" (UniqueName: \"kubernetes.io/projected/8438bd75-3a35-426f-9984-cfb6e2ff345f-kube-api-access-gjfh8\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.359885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.359641 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8438bd75-3a35-426f-9984-cfb6e2ff345f-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:09.723968 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.723933 2579 generic.go:356] "Generic (PLEG): container finished" podID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerID="6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88" exitCode=0 Jul 14 09:54:09.723968 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.723972 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" Jul 14 09:54:09.724167 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.724013 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" event={"ID":"8438bd75-3a35-426f-9984-cfb6e2ff345f","Type":"ContainerDied","Data":"6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88"} Jul 14 09:54:09.724167 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.724068 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8" event={"ID":"8438bd75-3a35-426f-9984-cfb6e2ff345f","Type":"ContainerDied","Data":"7242b501ffa2b1017fbc8a1a766f1e48f19ce74e6084aab0d0587784c0ac1cea"} Jul 14 09:54:09.724167 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.724090 2579 scope.go:117] "RemoveContainer" containerID="6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88" Jul 14 09:54:09.733148 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.733123 2579 scope.go:117] "RemoveContainer" containerID="7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2" Jul 14 09:54:09.741938 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.741899 2579 scope.go:117] "RemoveContainer" containerID="6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88" Jul 14 09:54:09.742195 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:54:09.742172 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88\": container with ID starting with 6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88 not found: ID does not exist" containerID="6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88" Jul 14 09:54:09.742304 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.742202 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88"} err="failed to get container status \"6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88\": rpc error: code = NotFound desc = could not find container \"6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88\": container with ID starting with 6859acc8cc7a3bf345ec0939407b82295808e576a4f51df313556540d14b2e88 not found: ID does not exist" Jul 14 09:54:09.742304 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.742220 2579 scope.go:117] "RemoveContainer" containerID="7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2" Jul 14 09:54:09.742500 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:54:09.742482 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2\": container with ID starting with 7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2 not found: ID does not exist" containerID="7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2" Jul 14 09:54:09.742567 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.742507 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2"} err="failed to get container status \"7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2\": rpc error: code = NotFound desc = could not find container \"7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2\": container with ID starting with 7d0e27be90d5992b61bded51e636194865576f8708c31b322f99205e7ebd82e2 not found: ID does not exist" Jul 14 09:54:09.742798 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.742780 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8"] Jul 14 09:54:09.746814 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:09.746795 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-40d96159-gw-openshift-default-6cd78dcc7f-djsd8"] Jul 14 09:54:10.164417 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.164382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:10.235966 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.235929 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:10.370824 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.370788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421062 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk"] Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421700 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421720 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421733 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a57b1877-827f-438d-902e-02f58a560d29" containerName="revoke-keys" Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421742 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a57b1877-827f-438d-902e-02f58a560d29" containerName="revoke-keys" Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421777 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" Jul 14 09:54:10.421818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421787 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" Jul 14 09:54:10.422302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421880 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" Jul 14 09:54:10.422302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421894 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" containerName="istio-proxy" Jul 14 09:54:10.422302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421903 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b3b59a-fd16-4e39-b4fa-d54c15d72d90" containerName="revoke-keys" Jul 14 09:54:10.422302 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.421915 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a57b1877-827f-438d-902e-02f58a560d29" containerName="revoke-keys" Jul 14 09:54:10.429725 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.429696 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.432421 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.432399 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ae87aa32-openshift-default-dockercfg-tqdbl\"" Jul 14 09:54:10.446036 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.446011 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk"] Jul 14 09:54:10.528858 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.528818 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:10.570690 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-certs\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.570892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-data\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.570892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-podinfo\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.570892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570737 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-token\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.570892 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-socket\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.571050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570892 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-envoy\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.571050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwdt2\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-kube-api-access-gwdt2\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.571050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.570979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-credential-socket\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.571050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.571014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istiod-ca-cert\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.671741 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-data\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.671741 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671743 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-podinfo\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-token\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671866 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-socket\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-envoy\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671940 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwdt2\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-kube-api-access-gwdt2\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.671978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-credential-socket\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672013 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istiod-ca-cert\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-certs\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672150 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-data\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-credential-socket\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672373 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-certs\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672626 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672517 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-socket\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.672774 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.672728 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istiod-ca-cert\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.674164 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.674146 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-envoy\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.674372 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.674353 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-podinfo\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.685834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.685808 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-token\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.685834 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.685826 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwdt2\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-kube-api-access-gwdt2\") pod \"e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.740951 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.740923 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:10.895045 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.895016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk"] Jul 14 09:54:10.897152 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:54:10.897127 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68fd8ac_a5c3_400b_9e37_89138034fa6e.slice/crio-f7ed41e739af9d7432eda677a94033344e91624f35901a8545d2fd14fb805c6c WatchSource:0}: Error finding container f7ed41e739af9d7432eda677a94033344e91624f35901a8545d2fd14fb805c6c: Status 404 returned error can't find the container with id f7ed41e739af9d7432eda677a94033344e91624f35901a8545d2fd14fb805c6c Jul 14 09:54:10.899246 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.899206 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:10.899333 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.899287 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:10.899389 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:10.899329 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:11.638824 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.638782 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8438bd75-3a35-426f-9984-cfb6e2ff345f" path="/var/lib/kubelet/pods/8438bd75-3a35-426f-9984-cfb6e2ff345f/volumes" Jul 14 09:54:11.643998 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.643969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:11.736429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.736398 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" event={"ID":"d68fd8ac-a5c3-400b-9e37-89138034fa6e","Type":"ContainerStarted","Data":"d2b3939f5bda25712e9ebb5f5ea228c74cba78b1683935a44d1d7296c3c3feb7"} Jul 14 09:54:11.736613 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.736455 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" event={"ID":"d68fd8ac-a5c3-400b-9e37-89138034fa6e","Type":"ContainerStarted","Data":"f7ed41e739af9d7432eda677a94033344e91624f35901a8545d2fd14fb805c6c"} Jul 14 09:54:11.737341 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.737319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:11.741386 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.741370 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:11.746094 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.746073 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:11.762895 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.762843 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" podStartSLOduration=1.7628273079999999 podStartE2EDuration="1.762827308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:54:11.760004302 +0000 UTC m=+1662.697854784" watchObservedRunningTime="2026-07-14 09:54:11.762827308 +0000 UTC m=+1662.700677777" Jul 14 09:54:11.831446 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.831410 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:11.947853 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:11.947765 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:12.284353 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:12.284322 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:12.384031 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:12.383989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:12.742021 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:12.741928 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:12.742786 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:12.742769 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:13.777388 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:13.777359 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:14.719616 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:14.719572 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:54:22.805052 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:22.805018 2579 generic.go:356] "Generic (PLEG): container finished" podID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerID="d2b3939f5bda25712e9ebb5f5ea228c74cba78b1683935a44d1d7296c3c3feb7" exitCode=0 Jul 14 09:54:22.805473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:22.805070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" event={"ID":"d68fd8ac-a5c3-400b-9e37-89138034fa6e","Type":"ContainerDied","Data":"d2b3939f5bda25712e9ebb5f5ea228c74cba78b1683935a44d1d7296c3c3feb7"} Jul 14 09:54:22.805473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:22.805462 2579 scope.go:117] "RemoveContainer" containerID="d2b3939f5bda25712e9ebb5f5ea228c74cba78b1683935a44d1d7296c3c3feb7" Jul 14 09:54:22.806447 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:22.806416 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:22.806544 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:22.806487 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:22.806544 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:22.806518 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:23.812109 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:23.812076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" event={"ID":"d68fd8ac-a5c3-400b-9e37-89138034fa6e","Type":"ContainerStarted","Data":"c102526ed2afbc415d0e56c1067aa4d94bebc78923ef2efd6bf6cc5099e5edd8"} Jul 14 09:54:24.741659 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:24.741621 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:24.742915 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:24.742891 2579 patch_prober.go:28] interesting pod/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 14 09:54:24.742990 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:24.742946 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 14 09:54:25.741620 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:25.741583 2579 patch_prober.go:28] interesting pod/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 14 09:54:25.742023 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:25.741637 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 14 09:54:26.742278 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:26.742236 2579 patch_prober.go:28] interesting pod/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 14 09:54:26.742740 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:26.742297 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 14 09:54:27.131486 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:27.131449 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:27.742006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:27.741971 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:27.746028 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:27.746007 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:27.832493 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:27.832469 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:31.789313 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:31.789279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:32.275212 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:32.275178 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:54:37.459862 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:37.459827 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:37.560050 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:37.560014 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:42.742397 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:42.742361 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk"] Jul 14 09:54:42.742811 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:42.742607 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" containerID="cri-o://c102526ed2afbc415d0e56c1067aa4d94bebc78923ef2efd6bf6cc5099e5edd8" gracePeriod=30 Jul 14 09:54:42.766309 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:42.766276 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:42.834294 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:42.834253 2579 patch_prober.go:28] interesting pod/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 09:54:42.834454 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:42.834324 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 09:54:43.043615 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.043579 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd"] Jul 14 09:54:43.047556 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.047531 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.050525 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.050503 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-376a5d8c-openshift-default-dockercfg-phzkm\"" Jul 14 09:54:43.064457 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.064429 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd"] Jul 14 09:54:43.097183 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.097144 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:43.138426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-envoy\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-credential-socket\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138484 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-data\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drtvs\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-kube-api-access-drtvs\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-certs\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-token\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138847 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istiod-ca-cert\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138847 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138654 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-podinfo\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.138847 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.138705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-socket\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239673 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-socket\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-envoy\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239720 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-credential-socket\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239741 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-data\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drtvs\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-kube-api-access-drtvs\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-certs\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-token\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.239913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.239907 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istiod-ca-cert\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.240292 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.240120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-podinfo\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.240292 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.240147 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-socket\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.240292 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.240188 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-credential-socket\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.240292 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.240279 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-data\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.240447 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.240324 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-certs\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.240803 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.240780 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istiod-ca-cert\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.242161 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.242135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-envoy\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.242570 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.242551 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-podinfo\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.248216 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.248187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-token\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.248312 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.248296 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drtvs\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-kube-api-access-drtvs\") pod \"e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.362143 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.362047 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:43.513012 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.512987 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd"] Jul 14 09:54:43.514859 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:54:43.514828 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc35d5ea_f6b6_447e_a514_a936be73b4e0.slice/crio-350601823ddb33610f64e16bab1f5ebe1ba37776f1b3f02a1c574548eb633d31 WatchSource:0}: Error finding container 350601823ddb33610f64e16bab1f5ebe1ba37776f1b3f02a1c574548eb633d31: Status 404 returned error can't find the container with id 350601823ddb33610f64e16bab1f5ebe1ba37776f1b3f02a1c574548eb633d31 Jul 14 09:54:43.517015 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.516978 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:43.517114 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.517055 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:43.517114 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.517098 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:43.911801 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.911765 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" event={"ID":"fc35d5ea-f6b6-447e-a514-a936be73b4e0","Type":"ContainerStarted","Data":"88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd"} Jul 14 09:54:43.912293 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.911826 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" event={"ID":"fc35d5ea-f6b6-447e-a514-a936be73b4e0","Type":"ContainerStarted","Data":"350601823ddb33610f64e16bab1f5ebe1ba37776f1b3f02a1c574548eb633d31"} Jul 14 09:54:43.932135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:43.932081 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podStartSLOduration=0.932061218 podStartE2EDuration="932.061218ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:54:43.931132409 +0000 UTC m=+1694.868982878" watchObservedRunningTime="2026-07-14 09:54:43.932061218 +0000 UTC m=+1694.869911685" Jul 14 09:54:44.308941 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.308903 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:44.362200 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.362158 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:44.367885 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.367849 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:44.383461 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.383433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:44.582697 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.582615 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:44.906269 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.906176 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:44.917517 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.917480 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:44.918322 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:44.918304 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:45.007461 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:45.007408 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:47.020432 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:47.020397 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:54:47.934785 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:47.934724 2579 generic.go:356] "Generic (PLEG): container finished" podID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerID="c102526ed2afbc415d0e56c1067aa4d94bebc78923ef2efd6bf6cc5099e5edd8" exitCode=0 Jul 14 09:54:47.934932 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:47.934799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" event={"ID":"d68fd8ac-a5c3-400b-9e37-89138034fa6e","Type":"ContainerDied","Data":"c102526ed2afbc415d0e56c1067aa4d94bebc78923ef2efd6bf6cc5099e5edd8"} Jul 14 09:54:47.935003 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:47.934938 2579 scope.go:117] "RemoveContainer" containerID="d2b3939f5bda25712e9ebb5f5ea228c74cba78b1683935a44d1d7296c3c3feb7" Jul 14 09:54:48.043449 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.043421 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092047 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-data\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092099 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-socket\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092142 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istiod-ca-cert\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092170 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-certs\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092215 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwdt2\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-kube-api-access-gwdt2\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092243 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-token\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092275 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-envoy\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092308 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-podinfo\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.092411 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-credential-socket\") pod \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\" (UID: \"d68fd8ac-a5c3-400b-9e37-89138034fa6e\") " Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.093132 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.093616 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.094157 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:48.094489 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.094456 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:48.095405 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.095366 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:48.097996 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.097959 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:54:48.098310 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.098275 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 09:54:48.099002 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.098973 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:54:48.099136 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.099061 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-kube-api-access-gwdt2" (OuterVolumeSpecName: "kube-api-access-gwdt2") pod "d68fd8ac-a5c3-400b-9e37-89138034fa6e" (UID: "d68fd8ac-a5c3-400b-9e37-89138034fa6e"). InnerVolumeSpecName "kube-api-access-gwdt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:54:48.194096 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194061 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194096 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194092 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194096 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194104 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194112 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194121 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194130 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwdt2\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-kube-api-access-gwdt2\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194138 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194145 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.194328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.194153 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d68fd8ac-a5c3-400b-9e37-89138034fa6e-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:54:48.942515 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.942485 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" Jul 14 09:54:48.942698 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.942487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk" event={"ID":"d68fd8ac-a5c3-400b-9e37-89138034fa6e","Type":"ContainerDied","Data":"f7ed41e739af9d7432eda677a94033344e91624f35901a8545d2fd14fb805c6c"} Jul 14 09:54:48.942698 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.942615 2579 scope.go:117] "RemoveContainer" containerID="c102526ed2afbc415d0e56c1067aa4d94bebc78923ef2efd6bf6cc5099e5edd8" Jul 14 09:54:48.965285 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.965251 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk"] Jul 14 09:54:48.968723 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:48.968688 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ae87aa32-openshift-default-58fd7c4f9f-r5sgk"] Jul 14 09:54:49.105032 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:49.104948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:49.205804 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:49.205561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:49.636520 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:49.636477 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" path="/var/lib/kubelet/pods/d68fd8ac-a5c3-400b-9e37-89138034fa6e/volumes" Jul 14 09:54:52.490923 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:52.490884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:54:54.975935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:54.975903 2579 generic.go:356] "Generic (PLEG): container finished" podID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerID="88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd" exitCode=0 Jul 14 09:54:54.976429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:54.975979 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" event={"ID":"fc35d5ea-f6b6-447e-a514-a936be73b4e0","Type":"ContainerDied","Data":"88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd"} Jul 14 09:54:54.976429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:54.976371 2579 scope.go:117] "RemoveContainer" containerID="88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd" Jul 14 09:54:54.977316 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:54.977291 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:54.977376 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:54.977353 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:54.977411 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:54.977384 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:54:55.983529 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:55.983491 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" event={"ID":"fc35d5ea-f6b6-447e-a514-a936be73b4e0","Type":"ContainerStarted","Data":"2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0"} Jul 14 09:54:56.363042 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:56.363004 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:54:56.364232 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:56.364198 2579 patch_prober.go:28] interesting pod/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 14 09:54:56.364386 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:56.364269 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 14 09:54:57.362986 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:57.362945 2579 patch_prober.go:28] interesting pod/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 14 09:54:57.363513 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:57.363016 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 14 09:54:58.362687 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:58.362642 2579 patch_prober.go:28] interesting pod/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 14 09:54:58.362918 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:58.362733 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 14 09:54:59.363169 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:59.363115 2579 patch_prober.go:28] interesting pod/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 14 09:54:59.363765 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:59.363718 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 14 09:54:59.917504 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:54:59.917470 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:55:00.366837 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:00.366808 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:55:00.367962 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:00.367942 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:55:04.928131 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:04.928080 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:55:05.025497 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:05.025458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:55:05.786254 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:05.786220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:55:05.888335 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:05.888307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:55:59.217313 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:55:59.217280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:02.222260 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:02.222222 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:56:03.703650 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:03.703623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:09.740526 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:09.740492 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:13.440222 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:13.440185 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:13.863207 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:13.863158 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd"] Jul 14 09:56:13.863441 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:13.863396 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" containerID="cri-o://2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0" gracePeriod=30 Jul 14 09:56:13.881670 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:13.881632 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:14.441453 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:14.441399 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:14.541501 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:14.541469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:14.642355 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:14.642309 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:15.369939 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:15.369902 2579 patch_prober.go:28] interesting pod/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 09:56:15.370113 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:15.369960 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 09:56:16.051550 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:16.051505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:19.127485 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.127462 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:56:19.230810 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.230777 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istiod-ca-cert\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.230977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.230844 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-credential-socket\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.230977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.230890 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-envoy\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.230977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.230942 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-certs\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.231135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231000 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drtvs\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-kube-api-access-drtvs\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.231135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231038 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-data\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.231135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231060 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-socket\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.231135 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231113 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-token\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.231343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231137 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-podinfo\") pod \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\" (UID: \"fc35d5ea-f6b6-447e-a514-a936be73b4e0\") " Jul 14 09:56:19.231343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231149 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:56:19.231343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231205 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:56:19.231343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231325 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:56:19.231343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231340 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:56:19.231593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231457 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.231593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231474 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.231593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231485 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.231593 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231498 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.231876 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.231850 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:56:19.233324 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.233289 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:56:19.233324 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.233312 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 09:56:19.233555 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.233538 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-kube-api-access-drtvs" (OuterVolumeSpecName: "kube-api-access-drtvs") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "kube-api-access-drtvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:56:19.233597 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.233553 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc35d5ea-f6b6-447e-a514-a936be73b4e0" (UID: "fc35d5ea-f6b6-447e-a514-a936be73b4e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:56:19.332192 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.332155 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.332192 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.332186 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drtvs\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-kube-api-access-drtvs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.332192 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.332196 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.332429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.332206 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.332429 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.332214 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc35d5ea-f6b6-447e-a514-a936be73b4e0-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:56:19.397978 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.397948 2579 generic.go:356] "Generic (PLEG): container finished" podID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerID="2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0" exitCode=0 Jul 14 09:56:19.398144 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.397997 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" Jul 14 09:56:19.398144 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.398019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" event={"ID":"fc35d5ea-f6b6-447e-a514-a936be73b4e0","Type":"ContainerDied","Data":"2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0"} Jul 14 09:56:19.398144 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.398090 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd" event={"ID":"fc35d5ea-f6b6-447e-a514-a936be73b4e0","Type":"ContainerDied","Data":"350601823ddb33610f64e16bab1f5ebe1ba37776f1b3f02a1c574548eb633d31"} Jul 14 09:56:19.398144 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.398117 2579 scope.go:117] "RemoveContainer" containerID="2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0" Jul 14 09:56:19.407133 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.407113 2579 scope.go:117] "RemoveContainer" containerID="88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd" Jul 14 09:56:19.415129 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.415113 2579 scope.go:117] "RemoveContainer" containerID="2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0" Jul 14 09:56:19.415380 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:56:19.415363 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0\": container with ID starting with 2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0 not found: ID does not exist" containerID="2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0" Jul 14 09:56:19.415443 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.415385 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0"} err="failed to get container status \"2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0\": rpc error: code = NotFound desc = could not find container \"2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0\": container with ID starting with 2bcd7da1ace25e206063d3f71d48ebe9a516a2075f8ef1cf8c83eb2ea018f5c0 not found: ID does not exist" Jul 14 09:56:19.415443 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.415401 2579 scope.go:117] "RemoveContainer" containerID="88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd" Jul 14 09:56:19.415621 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:56:19.415602 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd\": container with ID starting with 88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd not found: ID does not exist" containerID="88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd" Jul 14 09:56:19.415662 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.415627 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd"} err="failed to get container status \"88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd\": rpc error: code = NotFound desc = could not find container \"88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd\": container with ID starting with 88a5e23e4e1e6e8a91eb123659f9baf98692933a00fa799a525bf3737ff590bd not found: ID does not exist" Jul 14 09:56:19.422291 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.422267 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd"] Jul 14 09:56:19.427445 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.427425 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-376a5d8c-openshift-default-c88d858c-bh5kd"] Jul 14 09:56:19.636632 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:19.636589 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" path="/var/lib/kubelet/pods/fc35d5ea-f6b6-447e-a514-a936be73b4e0/volumes" Jul 14 09:56:21.734997 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:21.734918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:21.841052 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:21.841011 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:21.937833 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:21.937787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:22.039240 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:22.039208 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:22.422940 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:22.422866 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:29.650537 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:29.650511 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:56:29.654886 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:29.654867 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 09:56:30.798119 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:30.798090 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:32.528205 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:32.528169 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:48.435501 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:48.435470 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:54.857740 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:54.857710 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:55.005343 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:55.005283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:56:55.143450 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:56:55.143360 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:11.011718 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:11.011685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:17.052469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:17.052436 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:17.123350 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:17.123307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:17.223300 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:17.223270 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:17.498496 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:17.498406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:27.793819 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:27.793782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:28.608839 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:28.608809 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:34.665618 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:34.665583 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:34.807391 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:34.807359 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:34.978396 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:34.978314 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:35.090944 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:35.090915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:35.814602 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:35.814570 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:35.912507 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:35.912473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:37.534037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:37.534002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:38.318977 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:38.318944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:57:43.335624 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:43.335588 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:43.409934 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:43.409903 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:44.023021 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:44.022990 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:47.648176 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:47.648143 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:57:50.410167 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.410135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:50.475159 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.475120 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:50.644765 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.644712 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:50.978196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978165 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z"] Jul 14 09:57:50.978539 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978526 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" Jul 14 09:57:50.978581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978541 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" Jul 14 09:57:50.978581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978557 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" Jul 14 09:57:50.978581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978562 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" Jul 14 09:57:50.978581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978570 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" Jul 14 09:57:50.978581 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978576 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" Jul 14 09:57:50.978732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978587 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" Jul 14 09:57:50.978732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978592 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" Jul 14 09:57:50.978732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978658 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" Jul 14 09:57:50.978732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978664 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" Jul 14 09:57:50.978732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978672 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68fd8ac-a5c3-400b-9e37-89138034fa6e" containerName="istio-proxy" Jul 14 09:57:50.978732 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.978679 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc35d5ea-f6b6-447e-a514-a936be73b4e0" containerName="istio-proxy" Jul 14 09:57:50.981913 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.981897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:50.985978 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:50.985957 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dc470fea-openshift-default-dockercfg-4k7r5\"" Jul 14 09:57:51.014271 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.014246 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z"] Jul 14 09:57:51.019814 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.019792 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:51.060344 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-socket\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060344 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-credential-socket\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060385 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-token\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvwnc\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-kube-api-access-qvwnc\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-data\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istiod-ca-cert\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060543 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-podinfo\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-envoy\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.060699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.060617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-certs\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.161583 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-envoy\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.161777 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161608 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-certs\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.161777 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-socket\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.161777 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-credential-socket\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.161777 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-token\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvwnc\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-kube-api-access-qvwnc\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161844 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-data\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istiod-ca-cert\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162037 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.161913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-podinfo\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162320 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.162049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-certs\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162606 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.162579 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istiod-ca-cert\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162804 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.162636 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-socket\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162804 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.162800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-data\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.162982 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.162861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-credential-socket\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.164077 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.164056 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-envoy\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.164450 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.164433 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-podinfo\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.170110 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.170085 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-token\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.170193 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.170145 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvwnc\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-kube-api-access-qvwnc\") pod \"e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.292224 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.292188 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:51.359741 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.358904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:51.429940 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.429919 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z"] Jul 14 09:57:51.431545 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:57:51.431505 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a11c654_0eb5_43fc_976c_5346d63cf0f3.slice/crio-e42e46ba7cb0dd942d17ffa3961292f67787fd65f1290ace02bdf0b0151a38f1 WatchSource:0}: Error finding container e42e46ba7cb0dd942d17ffa3961292f67787fd65f1290ace02bdf0b0151a38f1: Status 404 returned error can't find the container with id e42e46ba7cb0dd942d17ffa3961292f67787fd65f1290ace02bdf0b0151a38f1 Jul 14 09:57:51.433553 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.433521 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:57:51.433654 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.433580 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:57:51.433654 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.433610 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:57:51.847465 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.847430 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" event={"ID":"5a11c654-0eb5-43fc-976c-5346d63cf0f3","Type":"ContainerStarted","Data":"62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695"} Jul 14 09:57:51.847643 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:51.847483 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" event={"ID":"5a11c654-0eb5-43fc-976c-5346d63cf0f3","Type":"ContainerStarted","Data":"e42e46ba7cb0dd942d17ffa3961292f67787fd65f1290ace02bdf0b0151a38f1"} Jul 14 09:57:52.292932 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:52.292901 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:52.297937 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:52.297914 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:52.321142 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:52.321092 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podStartSLOduration=2.321074909 podStartE2EDuration="2.321074909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:57:51.868000817 +0000 UTC m=+1882.805851288" watchObservedRunningTime="2026-07-14 09:57:52.321074909 +0000 UTC m=+1883.258925377" Jul 14 09:57:52.852507 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:52.852480 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:52.853426 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:52.853408 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:57:54.071087 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:54.071040 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:54.175473 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:54.175442 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:54.245284 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:54.245249 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:54.390396 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:54.390305 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:54.721009 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:54.720929 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:54.828351 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:54.828319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:58.539469 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:58.539431 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:59.179864 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:59.179828 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:59.277562 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:59.277529 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:57:59.428558 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:59.428524 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:57:59.654136 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:57:59.654079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:05.413578 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.413538 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:05.467995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.467964 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:05.765178 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.765144 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7"] Jul 14 09:58:05.768627 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.768605 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:05.771376 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.771354 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 09:58:05.771376 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.771354 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 09:58:05.776803 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.776777 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7"] Jul 14 09:58:05.901741 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:05.901712 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e761e99b-4616-4f66-b64a-0a0bed2cc613-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7\" (UID: \"e761e99b-4616-4f66-b64a-0a0bed2cc613\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:06.002974 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.002941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e761e99b-4616-4f66-b64a-0a0bed2cc613-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7\" (UID: \"e761e99b-4616-4f66-b64a-0a0bed2cc613\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:06.003818 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.003791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e761e99b-4616-4f66-b64a-0a0bed2cc613-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7\" (UID: \"e761e99b-4616-4f66-b64a-0a0bed2cc613\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:06.084884 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.084798 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:06.233149 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.233106 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7"] Jul 14 09:58:06.235822 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:58:06.235786 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode761e99b_4616_4f66_b64a_0a0bed2cc613.slice/crio-a1ab7548205ff9a73089665074a98acb3b80c8a42ed22e8f85ff2bde5efd0a07 WatchSource:0}: Error finding container a1ab7548205ff9a73089665074a98acb3b80c8a42ed22e8f85ff2bde5efd0a07: Status 404 returned error can't find the container with id a1ab7548205ff9a73089665074a98acb3b80c8a42ed22e8f85ff2bde5efd0a07 Jul 14 09:58:06.925351 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.925269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" event={"ID":"e761e99b-4616-4f66-b64a-0a0bed2cc613","Type":"ContainerStarted","Data":"a7a3f0e3bb53f55421f144a1e26a866ebb1c88832612537422519db9ca48be94"} Jul 14 09:58:06.925351 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.925320 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" event={"ID":"e761e99b-4616-4f66-b64a-0a0bed2cc613","Type":"ContainerStarted","Data":"a1ab7548205ff9a73089665074a98acb3b80c8a42ed22e8f85ff2bde5efd0a07"} Jul 14 09:58:06.943059 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:06.943005 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" podStartSLOduration=1.9429901429999998 podStartE2EDuration="1.942990143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:58:06.940894009 +0000 UTC m=+1897.878744482" watchObservedRunningTime="2026-07-14 09:58:06.942990143 +0000 UTC m=+1897.880840612" Jul 14 09:58:07.853698 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.853665 2579 patch_prober.go:28] interesting pod/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 14 09:58:07.853943 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.853721 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 14 09:58:07.931257 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.931223 2579 generic.go:356] "Generic (PLEG): container finished" podID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerID="62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695" exitCode=0 Jul 14 09:58:07.931699 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.931292 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" event={"ID":"5a11c654-0eb5-43fc-976c-5346d63cf0f3","Type":"ContainerDied","Data":"62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695"} Jul 14 09:58:07.931778 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.931727 2579 scope.go:117] "RemoveContainer" containerID="62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695" Jul 14 09:58:07.932805 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.932775 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:58:07.932907 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.932865 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:58:07.932971 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.932908 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:58:07.932971 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.932929 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" event={"ID":"e761e99b-4616-4f66-b64a-0a0bed2cc613","Type":"ContainerDied","Data":"a7a3f0e3bb53f55421f144a1e26a866ebb1c88832612537422519db9ca48be94"} Jul 14 09:58:07.933058 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:07.932908 2579 generic.go:356] "Generic (PLEG): container finished" podID="e761e99b-4616-4f66-b64a-0a0bed2cc613" containerID="a7a3f0e3bb53f55421f144a1e26a866ebb1c88832612537422519db9ca48be94" exitCode=0 Jul 14 09:58:08.946549 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:08.946514 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" event={"ID":"5a11c654-0eb5-43fc-976c-5346d63cf0f3","Type":"ContainerStarted","Data":"51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f"} Jul 14 09:58:09.126840 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.126814 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:09.233412 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.233321 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e761e99b-4616-4f66-b64a-0a0bed2cc613-maas-api-service-ca\") pod \"e761e99b-4616-4f66-b64a-0a0bed2cc613\" (UID: \"e761e99b-4616-4f66-b64a-0a0bed2cc613\") " Jul 14 09:58:09.233702 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.233681 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e761e99b-4616-4f66-b64a-0a0bed2cc613-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e761e99b-4616-4f66-b64a-0a0bed2cc613" (UID: "e761e99b-4616-4f66-b64a-0a0bed2cc613"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:58:09.293266 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.293227 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:58:09.293484 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.293465 2579 patch_prober.go:28] interesting pod/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 14 09:58:09.293530 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.293508 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 14 09:58:09.334855 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.334819 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e761e99b-4616-4f66-b64a-0a0bed2cc613-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:09.953681 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.953602 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" Jul 14 09:58:09.953681 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.953645 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7" event={"ID":"e761e99b-4616-4f66-b64a-0a0bed2cc613","Type":"ContainerDied","Data":"a1ab7548205ff9a73089665074a98acb3b80c8a42ed22e8f85ff2bde5efd0a07"} Jul 14 09:58:09.954149 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:09.953690 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1ab7548205ff9a73089665074a98acb3b80c8a42ed22e8f85ff2bde5efd0a07" Jul 14 09:58:10.292904 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:10.292868 2579 patch_prober.go:28] interesting pod/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 14 09:58:10.293089 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:10.292920 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 14 09:58:11.293459 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:11.293417 2579 patch_prober.go:28] interesting pod/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 14 09:58:11.293882 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:11.293507 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 14 09:58:12.292991 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:12.292951 2579 patch_prober.go:28] interesting pod/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 14 09:58:12.293179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:12.293023 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 14 09:58:13.297020 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:13.296992 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:58:13.297432 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:13.297248 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:58:13.298006 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:13.297989 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:58:16.337299 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:16.337266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:19.215266 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:19.215229 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:58:20.859956 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:20.859918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:26.958375 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:26.958340 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:27.058518 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:27.058482 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:31.558342 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:31.558306 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z"] Jul 14 09:58:31.558835 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:31.558557 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" containerID="cri-o://51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f" gracePeriod=30 Jul 14 09:58:31.581894 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:31.581862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:31.810536 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:31.810456 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:31.939922 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:31.939880 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:32.050004 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:32.049970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:32.177130 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:32.177043 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:36.818179 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.818155 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:58:36.877290 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877203 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-envoy\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877290 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877242 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-certs\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877290 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877283 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-token\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877313 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvwnc\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-kube-api-access-qvwnc\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877342 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-socket\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877426 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istiod-ca-cert\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877454 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-podinfo\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877486 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-data\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877531 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-credential-socket\") pod \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\" (UID: \"5a11c654-0eb5-43fc-976c-5346d63cf0f3\") " Jul 14 09:58:36.877905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877645 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:58:36.877905 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877729 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:58:36.878010 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877924 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:58:36.878010 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877936 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.878010 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.877957 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.879014 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.878157 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:58:36.879014 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.878970 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:58:36.879526 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.879505 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:58:36.879782 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.879765 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-kube-api-access-qvwnc" (OuterVolumeSpecName: "kube-api-access-qvwnc") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "kube-api-access-qvwnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:58:36.879935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.879907 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:58:36.879935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.879917 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a11c654-0eb5-43fc-976c-5346d63cf0f3" (UID: "5a11c654-0eb5-43fc-976c-5346d63cf0f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 09:58:36.978947 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978914 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.978947 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978942 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.978947 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978952 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.979196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978960 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.979196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978968 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvwnc\" (UniqueName: \"kubernetes.io/projected/5a11c654-0eb5-43fc-976c-5346d63cf0f3-kube-api-access-qvwnc\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.979196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978976 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:36.979196 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:36.978984 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a11c654-0eb5-43fc-976c-5346d63cf0f3-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:58:37.096250 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.096215 2579 generic.go:356] "Generic (PLEG): container finished" podID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerID="51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f" exitCode=0 Jul 14 09:58:37.096451 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.096264 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" Jul 14 09:58:37.096451 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.096302 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" event={"ID":"5a11c654-0eb5-43fc-976c-5346d63cf0f3","Type":"ContainerDied","Data":"51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f"} Jul 14 09:58:37.096451 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.096370 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z" event={"ID":"5a11c654-0eb5-43fc-976c-5346d63cf0f3","Type":"ContainerDied","Data":"e42e46ba7cb0dd942d17ffa3961292f67787fd65f1290ace02bdf0b0151a38f1"} Jul 14 09:58:37.096451 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.096401 2579 scope.go:117] "RemoveContainer" containerID="51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f" Jul 14 09:58:37.105552 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.105534 2579 scope.go:117] "RemoveContainer" containerID="62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695" Jul 14 09:58:37.113522 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.113503 2579 scope.go:117] "RemoveContainer" containerID="51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f" Jul 14 09:58:37.113796 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:58:37.113777 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f\": container with ID starting with 51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f not found: ID does not exist" containerID="51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f" Jul 14 09:58:37.113867 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.113804 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f"} err="failed to get container status \"51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f\": rpc error: code = NotFound desc = could not find container \"51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f\": container with ID starting with 51b707c64806cccf4da315f7c2fb2878086e67bab945bbceee7bb94cb68c307f not found: ID does not exist" Jul 14 09:58:37.113867 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.113820 2579 scope.go:117] "RemoveContainer" containerID="62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695" Jul 14 09:58:37.114059 ip-10-0-139-42 kubenswrapper[2579]: E0714 09:58:37.114045 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695\": container with ID starting with 62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695 not found: ID does not exist" containerID="62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695" Jul 14 09:58:37.114096 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.114062 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695"} err="failed to get container status \"62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695\": rpc error: code = NotFound desc = could not find container \"62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695\": container with ID starting with 62251be6e40f6f0de3f65ee1ed35f1d719c577e381c6c6e2cfad5f48e6c20695 not found: ID does not exist" Jul 14 09:58:37.119256 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.119234 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z"] Jul 14 09:58:37.125136 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.125113 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dc470fea-openshift-default-67bbd8fd7f-7nk7z"] Jul 14 09:58:37.636247 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.636209 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" path="/var/lib/kubelet/pods/5a11c654-0eb5-43fc-976c-5346d63cf0f3/volumes" Jul 14 09:58:37.966483 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:37.966396 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:38.035940 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:38.035900 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:38.125458 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:38.125427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:38.234654 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:38.234570 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:38.342931 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:38.342900 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:44.113526 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:44.113491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:44.174881 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:44.174849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:50.790092 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:50.790056 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:51.077628 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077542 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br"] Jul 14 09:58:51.077945 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077933 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" Jul 14 09:58:51.077995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077948 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" Jul 14 09:58:51.077995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077955 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e761e99b-4616-4f66-b64a-0a0bed2cc613" containerName="revoke-keys" Jul 14 09:58:51.077995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077961 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e761e99b-4616-4f66-b64a-0a0bed2cc613" containerName="revoke-keys" Jul 14 09:58:51.077995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077974 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" Jul 14 09:58:51.077995 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.077981 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" Jul 14 09:58:51.078153 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.078043 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" Jul 14 09:58:51.078153 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.078049 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a11c654-0eb5-43fc-976c-5346d63cf0f3" containerName="istio-proxy" Jul 14 09:58:51.078153 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.078058 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e761e99b-4616-4f66-b64a-0a0bed2cc613" containerName="revoke-keys" Jul 14 09:58:51.082587 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.082562 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.085067 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.085041 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b2b30ce5-openshift-default-dockercfg-66hfs\"" Jul 14 09:58:51.097607 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.097580 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br"] Jul 14 09:58:51.123502 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.123473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:51.209823 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.209786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-podinfo\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210036 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.209842 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqlp4\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-kube-api-access-pqlp4\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210036 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.209875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-data\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210036 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.209916 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05fbe04-081a-44c2-a800-fe2e63fe9353-istiod-ca-cert\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210036 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.209944 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-socket\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210036 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.209982 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-certs\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210228 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.210067 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-token\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210228 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.210105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-envoy\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.210228 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.210171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-credential-socket\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311162 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-podinfo\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311162 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311153 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqlp4\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-kube-api-access-pqlp4\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-data\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05fbe04-081a-44c2-a800-fe2e63fe9353-istiod-ca-cert\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311387 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-socket\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311439 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311436 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-certs\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311651 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311477 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-token\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311651 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311506 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-envoy\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311651 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311573 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-credential-socket\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311861 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311788 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-socket\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311861 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-data\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.311861 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311786 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-certs\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.312023 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.311960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05fbe04-081a-44c2-a800-fe2e63fe9353-istiod-ca-cert\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.312109 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.312090 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-credential-socket\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.313797 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.313770 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-envoy\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.313909 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.313879 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-podinfo\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.321809 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.321779 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-token\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.321942 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.321877 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqlp4\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-kube-api-access-pqlp4\") pod \"e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.392833 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.392713 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:51.532935 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.532904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:51.550632 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.550603 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br"] Jul 14 09:58:51.552379 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:58:51.552353 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05fbe04_081a_44c2_a800_fe2e63fe9353.slice/crio-eed625ad24510ec55c8ff47eaa627b6b846e83644adf801dc7e8e1ff8e5535fd WatchSource:0}: Error finding container eed625ad24510ec55c8ff47eaa627b6b846e83644adf801dc7e8e1ff8e5535fd: Status 404 returned error can't find the container with id eed625ad24510ec55c8ff47eaa627b6b846e83644adf801dc7e8e1ff8e5535fd Jul 14 09:58:51.554427 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.554393 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:58:51.554488 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.554459 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:58:51.554525 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:51.554497 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:58:52.172641 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:52.172597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" event={"ID":"b05fbe04-081a-44c2-a800-fe2e63fe9353","Type":"ContainerStarted","Data":"d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75"} Jul 14 09:58:52.173109 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:52.172706 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" event={"ID":"b05fbe04-081a-44c2-a800-fe2e63fe9353","Type":"ContainerStarted","Data":"eed625ad24510ec55c8ff47eaa627b6b846e83644adf801dc7e8e1ff8e5535fd"} Jul 14 09:58:52.194965 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:52.194901 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podStartSLOduration=1.194883318 podStartE2EDuration="1.194883318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:58:52.193918407 +0000 UTC m=+1943.131768881" watchObservedRunningTime="2026-07-14 09:58:52.194883318 +0000 UTC m=+1943.132733789" Jul 14 09:58:52.393826 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:52.393788 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:52.399625 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:52.399594 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:53.178062 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:53.178023 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:53.179027 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:53.179009 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:58:54.082918 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:54.082884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:54.193175 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:54.193147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:54.292576 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:54.292530 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:54.389416 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:54.389315 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:54.924677 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:54.924643 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:55.023911 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:55.023882 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:58.623879 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:58.623843 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:58:59.617055 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:58:59.617016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:59:03.980771 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:03.979481 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:59:08.179958 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.179920 2579 patch_prober.go:28] interesting pod/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 14 09:59:08.180374 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.179977 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 14 09:59:08.257066 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.257034 2579 generic.go:356] "Generic (PLEG): container finished" podID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerID="d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75" exitCode=0 Jul 14 09:59:08.257223 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.257109 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" event={"ID":"b05fbe04-081a-44c2-a800-fe2e63fe9353","Type":"ContainerDied","Data":"d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75"} Jul 14 09:59:08.257524 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.257509 2579 scope.go:117] "RemoveContainer" containerID="d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75" Jul 14 09:59:08.258498 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.258471 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:59:08.258561 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.258536 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:59:08.258600 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:08.258566 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 09:59:09.268678 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:09.268632 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" event={"ID":"b05fbe04-081a-44c2-a800-fe2e63fe9353","Type":"ContainerStarted","Data":"89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98"} Jul 14 09:59:09.393213 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:09.393181 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:59:09.393409 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:09.393387 2579 patch_prober.go:28] interesting pod/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 14 09:59:09.393460 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:09.393433 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 14 09:59:10.393293 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:10.393258 2579 patch_prober.go:28] interesting pod/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 14 09:59:10.393684 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:10.393318 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 14 09:59:11.026054 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:11.026013 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:59:11.393979 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:11.393885 2579 patch_prober.go:28] interesting pod/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 14 09:59:11.393979 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:11.393943 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 14 09:59:12.393644 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:12.393610 2579 patch_prober.go:28] interesting pod/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 14 09:59:12.393877 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:12.393670 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 14 09:59:13.397150 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:13.397119 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:59:13.397587 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:13.397473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:59:13.398139 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:13.398122 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 09:59:24.826820 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:24.826729 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:59:30.131917 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:30.131884 2579 scope.go:117] "RemoveContainer" containerID="c6b248cbdc184ae423850bbb3b475839eb9afe0ffc560c28ddafba8d5e927a46" Jul 14 09:59:31.651400 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.651369 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr"] Jul 14 09:59:31.656266 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.656248 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:31.659356 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.659324 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 09:59:31.659582 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.659561 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 09:59:31.664243 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.664211 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr"] Jul 14 09:59:31.789223 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.789186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c51939-4996-4a2b-90c1-604123271f48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr\" (UID: \"50c51939-4996-4a2b-90c1-604123271f48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:31.891245 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.891187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c51939-4996-4a2b-90c1-604123271f48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr\" (UID: \"50c51939-4996-4a2b-90c1-604123271f48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:31.892328 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.892297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c51939-4996-4a2b-90c1-604123271f48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr\" (UID: \"50c51939-4996-4a2b-90c1-604123271f48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:31.968385 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:31.968286 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:32.105531 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:32.105505 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr"] Jul 14 09:59:32.107740 ip-10-0-139-42 kubenswrapper[2579]: W0714 09:59:32.107708 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c51939_4996_4a2b_90c1_604123271f48.slice/crio-c20ccbf41aa79c5970c426b546e079147a4df476603a39e60c29909b3a1ef22d WatchSource:0}: Error finding container c20ccbf41aa79c5970c426b546e079147a4df476603a39e60c29909b3a1ef22d: Status 404 returned error can't find the container with id c20ccbf41aa79c5970c426b546e079147a4df476603a39e60c29909b3a1ef22d Jul 14 09:59:32.400059 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:32.400015 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" event={"ID":"50c51939-4996-4a2b-90c1-604123271f48","Type":"ContainerStarted","Data":"1006d5113747fdf64a3bcfc177585a646324677ad7050b3834dd04fa5bb2cd4c"} Jul 14 09:59:32.400238 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:32.400086 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" event={"ID":"50c51939-4996-4a2b-90c1-604123271f48","Type":"ContainerStarted","Data":"c20ccbf41aa79c5970c426b546e079147a4df476603a39e60c29909b3a1ef22d"} Jul 14 09:59:32.417795 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:32.417735 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" podStartSLOduration=1.417720045 podStartE2EDuration="1.417720045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:59:32.416149767 +0000 UTC m=+1983.354000239" watchObservedRunningTime="2026-07-14 09:59:32.417720045 +0000 UTC m=+1983.355570514" Jul 14 09:59:33.406501 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:33.406462 2579 generic.go:356] "Generic (PLEG): container finished" podID="50c51939-4996-4a2b-90c1-604123271f48" containerID="1006d5113747fdf64a3bcfc177585a646324677ad7050b3834dd04fa5bb2cd4c" exitCode=0 Jul 14 09:59:33.406926 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:33.406544 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" event={"ID":"50c51939-4996-4a2b-90c1-604123271f48","Type":"ContainerDied","Data":"1006d5113747fdf64a3bcfc177585a646324677ad7050b3834dd04fa5bb2cd4c"} Jul 14 09:59:34.548768 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:34.548730 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:34.615208 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:34.615178 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c51939-4996-4a2b-90c1-604123271f48-maas-api-service-ca\") pod \"50c51939-4996-4a2b-90c1-604123271f48\" (UID: \"50c51939-4996-4a2b-90c1-604123271f48\") " Jul 14 09:59:34.615540 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:34.615518 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c51939-4996-4a2b-90c1-604123271f48-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50c51939-4996-4a2b-90c1-604123271f48" (UID: "50c51939-4996-4a2b-90c1-604123271f48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:59:34.716154 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:34.716063 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c51939-4996-4a2b-90c1-604123271f48-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 09:59:35.418294 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:35.418264 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" Jul 14 09:59:35.418501 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:35.418295 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr" event={"ID":"50c51939-4996-4a2b-90c1-604123271f48","Type":"ContainerDied","Data":"c20ccbf41aa79c5970c426b546e079147a4df476603a39e60c29909b3a1ef22d"} Jul 14 09:59:35.418501 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:35.418336 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c20ccbf41aa79c5970c426b546e079147a4df476603a39e60c29909b3a1ef22d" Jul 14 09:59:39.026112 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:39.026076 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:59:39.128454 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:39.128417 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:59:42.114580 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:42.114552 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:59:45.718570 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:45.718533 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 09:59:46.733340 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:46.733306 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 09:59:51.810883 ip-10-0-139-42 kubenswrapper[2579]: I0714 09:59:51.810849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:06.025060 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:06.025024 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:06.124462 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:06.124432 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:28.227922 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:28.227883 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:28.942730 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:28.942695 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:34.477630 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:34.477587 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:35.244212 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:35.244170 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:36.291204 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:36.291129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:38.917292 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:38.917259 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br"] Jul 14 10:00:38.917661 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:38.917512 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" containerID="cri-o://89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98" gracePeriod=30 Jul 14 10:00:38.942624 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:38.942582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:39.157499 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:39.157469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:40.186179 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:40.186122 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:40.275414 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:40.275384 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:40.380544 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:40.380503 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:41.077572 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:41.077540 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:41.174622 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:41.174589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:41.275790 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:41.275734 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:42.620389 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:42.620356 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:00:42.727610 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:42.727576 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:43.399691 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:43.399661 2579 patch_prober.go:28] interesting pod/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 10:00:43.399878 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:43.399710 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 10:00:44.172399 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.172376 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 10:00:44.240306 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240216 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqlp4\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-kube-api-access-pqlp4\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240306 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240276 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-envoy\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240306 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240295 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-token\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240594 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240317 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-credential-socket\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240594 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240340 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-certs\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240594 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240377 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-podinfo\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240594 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240405 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-data\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240594 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240432 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05fbe04-081a-44c2-a800-fe2e63fe9353-istiod-ca-cert\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240594 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240484 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-socket\") pod \"b05fbe04-081a-44c2-a800-fe2e63fe9353\" (UID: \"b05fbe04-081a-44c2-a800-fe2e63fe9353\") " Jul 14 10:00:44.240956 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240603 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:00:44.240956 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240774 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.241066 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.240947 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:00:44.241161 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.241122 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b05fbe04-081a-44c2-a800-fe2e63fe9353-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:00:44.241381 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.241355 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:00:44.241659 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.241633 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-data" (OuterVolumeSpecName: "istio-data") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:00:44.242857 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.242823 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-token" (OuterVolumeSpecName: "istio-token") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:00:44.242958 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.242849 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-kube-api-access-pqlp4" (OuterVolumeSpecName: "kube-api-access-pqlp4") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "kube-api-access-pqlp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:00:44.242958 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.242890 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:00:44.243072 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.243041 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b05fbe04-081a-44c2-a800-fe2e63fe9353" (UID: "b05fbe04-081a-44c2-a800-fe2e63fe9353"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:00:44.341655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341619 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341649 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqlp4\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-kube-api-access-pqlp4\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341662 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341922 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341670 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341922 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341678 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341922 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341685 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341922 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341692 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b05fbe04-081a-44c2-a800-fe2e63fe9353-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.341922 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.341699 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b05fbe04-081a-44c2-a800-fe2e63fe9353-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:00:44.779129 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.779098 2579 generic.go:356] "Generic (PLEG): container finished" podID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerID="89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98" exitCode=0 Jul 14 10:00:44.779308 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.779146 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" Jul 14 10:00:44.779308 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.779185 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" event={"ID":"b05fbe04-081a-44c2-a800-fe2e63fe9353","Type":"ContainerDied","Data":"89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98"} Jul 14 10:00:44.779308 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.779235 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br" event={"ID":"b05fbe04-081a-44c2-a800-fe2e63fe9353","Type":"ContainerDied","Data":"eed625ad24510ec55c8ff47eaa627b6b846e83644adf801dc7e8e1ff8e5535fd"} Jul 14 10:00:44.779308 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.779258 2579 scope.go:117] "RemoveContainer" containerID="89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98" Jul 14 10:00:44.788698 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.788678 2579 scope.go:117] "RemoveContainer" containerID="d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75" Jul 14 10:00:44.796693 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.796677 2579 scope.go:117] "RemoveContainer" containerID="89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98" Jul 14 10:00:44.796936 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:00:44.796910 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98\": container with ID starting with 89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98 not found: ID does not exist" containerID="89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98" Jul 14 10:00:44.796995 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.796946 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98"} err="failed to get container status \"89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98\": rpc error: code = NotFound desc = could not find container \"89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98\": container with ID starting with 89fbd325249b2d7a230fccb63a547eaba6165748df7dd3bf2fbad3f8a4550a98 not found: ID does not exist" Jul 14 10:00:44.796995 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.796965 2579 scope.go:117] "RemoveContainer" containerID="d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75" Jul 14 10:00:44.797176 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:00:44.797157 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75\": container with ID starting with d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75 not found: ID does not exist" containerID="d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75" Jul 14 10:00:44.797223 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.797183 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75"} err="failed to get container status \"d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75\": rpc error: code = NotFound desc = could not find container \"d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75\": container with ID starting with d36fbc3cfb49637ea7866a8d983540a7512f64231713bcd0d8bb1b86b4810e75 not found: ID does not exist" Jul 14 10:00:44.801089 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.801066 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br"] Jul 14 10:00:44.807639 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:44.807619 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b2b30ce5-openshift-default-685b467f47-tp7br"] Jul 14 10:00:45.636860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:45.636823 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" path="/var/lib/kubelet/pods/b05fbe04-081a-44c2-a800-fe2e63fe9353/volumes" Jul 14 10:00:48.336500 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:48.336465 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:48.401543 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:48.401503 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:48.982989 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:48.982956 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:52.846942 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:52.846869 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:52.947881 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:52.947847 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:55.342434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:55.342397 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:55.457830 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:55.457801 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:55.568766 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:55.568702 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:00:57.627357 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:00:57.627297 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:01:06.053897 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:06.053863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:06.382930 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:06.382845 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:28.140871 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:28.140836 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:29.679055 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:29.679030 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:01:29.684208 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:29.684177 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:01:33.160917 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.160884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:33.261986 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.261949 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:33.532655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.532628 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:33.807995 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.807912 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8"] Jul 14 10:01:33.808316 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808303 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" Jul 14 10:01:33.808356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808318 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" Jul 14 10:01:33.808356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808330 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" Jul 14 10:01:33.808356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808336 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" Jul 14 10:01:33.808356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808348 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50c51939-4996-4a2b-90c1-604123271f48" containerName="revoke-keys" Jul 14 10:01:33.808356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808354 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c51939-4996-4a2b-90c1-604123271f48" containerName="revoke-keys" Jul 14 10:01:33.808510 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808418 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" Jul 14 10:01:33.808510 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808426 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="50c51939-4996-4a2b-90c1-604123271f48" containerName="revoke-keys" Jul 14 10:01:33.808510 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.808431 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05fbe04-081a-44c2-a800-fe2e63fe9353" containerName="istio-proxy" Jul 14 10:01:33.811653 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.811634 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.814120 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.814098 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-06e8a8-openshift-default-dockercfg-hb5pv\"" Jul 14 10:01:33.823591 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.823570 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8"] Jul 14 10:01:33.835011 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.834989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:33.882466 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9a53af0-84a9-45db-bd34-167131532f94-istiod-ca-cert\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882466 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882468 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-certs\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882666 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882570 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9a53af0-84a9-45db-bd34-167131532f94-istio-podinfo\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882666 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-socket\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882666 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-istio-token\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882816 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882666 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-envoy\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882816 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-credential-socket\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882816 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882710 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-data\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.882816 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.882726 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzhfx\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-kube-api-access-zzhfx\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.931131 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.931102 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:33.983260 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983227 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9a53af0-84a9-45db-bd34-167131532f94-istio-podinfo\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983429 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-socket\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983429 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-istio-token\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983548 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983468 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-envoy\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983548 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-credential-socket\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983662 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-data\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983662 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983582 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzhfx\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-kube-api-access-zzhfx\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983780 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983698 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9a53af0-84a9-45db-bd34-167131532f94-istiod-ca-cert\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983780 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-certs\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.983890 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.983788 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-socket\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.984277 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.984180 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-credential-socket\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.984277 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.984244 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-data\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.984481 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.984308 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-certs\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.984481 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.984435 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9a53af0-84a9-45db-bd34-167131532f94-istiod-ca-cert\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.986151 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.986127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9a53af0-84a9-45db-bd34-167131532f94-istio-podinfo\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.986253 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.986241 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-envoy\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.992264 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.992241 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-istio-token\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:33.992385 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:33.992271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzhfx\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-kube-api-access-zzhfx\") pod \"e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:34.122262 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:34.122174 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:34.268770 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:34.268715 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8"] Jul 14 10:01:34.270672 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:01:34.270647 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a53af0_84a9_45db_bd34_167131532f94.slice/crio-b22e81ad7a035a4dec21df00d5fecfc77a0a4a32d31faf395eac296ba694f9ec WatchSource:0}: Error finding container b22e81ad7a035a4dec21df00d5fecfc77a0a4a32d31faf395eac296ba694f9ec: Status 404 returned error can't find the container with id b22e81ad7a035a4dec21df00d5fecfc77a0a4a32d31faf395eac296ba694f9ec Jul 14 10:01:34.273337 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:34.273302 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:34.273441 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:34.273370 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:34.273441 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:34.273397 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:35.027520 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:35.027484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" event={"ID":"e9a53af0-84a9-45db-bd34-167131532f94","Type":"ContainerStarted","Data":"633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc"} Jul 14 10:01:35.027724 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:35.027538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" event={"ID":"e9a53af0-84a9-45db-bd34-167131532f94","Type":"ContainerStarted","Data":"b22e81ad7a035a4dec21df00d5fecfc77a0a4a32d31faf395eac296ba694f9ec"} Jul 14 10:01:35.050212 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:35.050157 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podStartSLOduration=2.050139539 podStartE2EDuration="2.050139539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:35.047558557 +0000 UTC m=+2105.985409026" watchObservedRunningTime="2026-07-14 10:01:35.050139539 +0000 UTC m=+2105.987990010" Jul 14 10:01:35.122593 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:35.122561 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:35.127773 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:35.127729 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:36.033429 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:36.033391 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:36.034310 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:36.034292 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:36.941266 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:36.941230 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:38.054476 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:38.054446 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:38.157982 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:38.157936 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:41.466539 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:41.466499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:41.725051 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:41.724965 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:41.993700 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:41.993621 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg"] Jul 14 10:01:41.997604 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:41.997587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.000064 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.000043 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d7ae40-openshift-default-dockercfg-fh2zj\"" Jul 14 10:01:42.010211 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.010185 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg"] Jul 14 10:01:42.021700 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.021679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:42.055888 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.055863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-credential-socket\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.055900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-istio-token\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.055926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/626f1d3d-3d95-4963-a121-cc8833f84506-istiod-ca-cert\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.055947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klwxp\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-kube-api-access-klwxp\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.055991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-socket\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056197 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.056021 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-certs\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056197 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.056038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-data\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056197 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.056059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-envoy\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.056332 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.056243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/626f1d3d-3d95-4963-a121-cc8833f84506-istio-podinfo\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157286 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/626f1d3d-3d95-4963-a121-cc8833f84506-istio-podinfo\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157286 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-credential-socket\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157501 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-istio-token\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157539 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/626f1d3d-3d95-4963-a121-cc8833f84506-istiod-ca-cert\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157539 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157534 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klwxp\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-kube-api-access-klwxp\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157689 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-socket\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157801 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-certs\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157801 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157675 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-credential-socket\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157801 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-data\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.157980 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157819 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-envoy\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.158037 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.157994 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-socket\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.158115 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.158093 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-certs\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.158199 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.158177 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/626f1d3d-3d95-4963-a121-cc8833f84506-istiod-ca-cert\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.158268 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.158200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-data\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.160071 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.160040 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/626f1d3d-3d95-4963-a121-cc8833f84506-istio-podinfo\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.160200 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.160155 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-envoy\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.165027 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.165006 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-istio-token\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.165144 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.165102 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klwxp\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-kube-api-access-klwxp\") pod \"e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.223494 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.223465 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:42.309904 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.309872 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:42.451895 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.451872 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg"] Jul 14 10:01:42.453781 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:01:42.453738 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626f1d3d_3d95_4963_a121_cc8833f84506.slice/crio-a1f2b85db572cc0230a11011b4dc4b766e3df94a05470cbd054002babb079751 WatchSource:0}: Error finding container a1f2b85db572cc0230a11011b4dc4b766e3df94a05470cbd054002babb079751: Status 404 returned error can't find the container with id a1f2b85db572cc0230a11011b4dc4b766e3df94a05470cbd054002babb079751 Jul 14 10:01:42.455949 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.455916 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:42.456041 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.455991 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:42.456041 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:42.456021 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:43.027308 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:43.027276 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:43.075706 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:43.075672 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" event={"ID":"626f1d3d-3d95-4963-a121-cc8833f84506","Type":"ContainerStarted","Data":"1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416"} Jul 14 10:01:43.075907 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:43.075721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" event={"ID":"626f1d3d-3d95-4963-a121-cc8833f84506","Type":"ContainerStarted","Data":"a1f2b85db572cc0230a11011b4dc4b766e3df94a05470cbd054002babb079751"} Jul 14 10:01:43.097453 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:43.097378 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podStartSLOduration=2.097362732 podStartE2EDuration="2.097362732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:43.095515582 +0000 UTC m=+2114.033366053" watchObservedRunningTime="2026-07-14 10:01:43.097362732 +0000 UTC m=+2114.035213202" Jul 14 10:01:43.310316 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:43.310233 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:43.315168 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:43.315143 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:44.080419 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:44.080389 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:44.081341 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:44.081320 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:01:44.971828 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:44.971793 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:45.050173 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:45.050141 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:45.133555 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:45.133523 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:45.426914 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:45.426872 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:45.716954 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:45.716877 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:01:45.718336 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:45.718307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:45.824663 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:45.824620 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:49.448673 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:49.448631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:52.075948 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:52.075904 2579 patch_prober.go:28] interesting pod/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 10:01:52.076364 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:52.075977 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 10:01:54.130416 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:54.130380 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9a53af0-84a9-45db-bd34-167131532f94" containerID="633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc" exitCode=0 Jul 14 10:01:54.130965 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:54.130428 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" event={"ID":"e9a53af0-84a9-45db-bd34-167131532f94","Type":"ContainerDied","Data":"633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc"} Jul 14 10:01:54.130965 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:54.130914 2579 scope.go:117] "RemoveContainer" containerID="633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc" Jul 14 10:01:54.131962 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:54.131933 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:54.132030 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:54.132012 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:54.132066 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:54.132043 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:01:55.137151 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:55.137117 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" event={"ID":"e9a53af0-84a9-45db-bd34-167131532f94","Type":"ContainerStarted","Data":"e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a"} Jul 14 10:01:56.122768 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:56.122712 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:56.123124 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:56.123102 2579 patch_prober.go:28] interesting pod/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 14 10:01:56.123257 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:56.123155 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 14 10:01:57.123029 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:57.123000 2579 patch_prober.go:28] interesting pod/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 14 10:01:57.123531 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:57.123269 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 14 10:01:58.123066 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:58.123018 2579 patch_prober.go:28] interesting pod/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 14 10:01:58.123481 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:58.123086 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 14 10:01:58.724815 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:58.724786 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:58.824913 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:58.824882 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:01:59.126097 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:59.126061 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:59.126579 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:59.126441 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:01:59.126986 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:01:59.126968 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:02:00.126655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:00.126619 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:02:02.177213 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:02.177133 2579 generic.go:356] "Generic (PLEG): container finished" podID="626f1d3d-3d95-4963-a121-cc8833f84506" containerID="1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416" exitCode=0 Jul 14 10:02:02.177213 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:02.177197 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" event={"ID":"626f1d3d-3d95-4963-a121-cc8833f84506","Type":"ContainerDied","Data":"1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416"} Jul 14 10:02:02.177631 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:02.177588 2579 scope.go:117] "RemoveContainer" containerID="1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416" Jul 14 10:02:02.178603 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:02.178574 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:02:02.178667 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:02.178641 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:02:02.178705 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:02.178673 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:02:03.184396 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:03.184360 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" event={"ID":"626f1d3d-3d95-4963-a121-cc8833f84506","Type":"ContainerStarted","Data":"2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add"} Jul 14 10:02:03.310593 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:03.310553 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:02:03.310858 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:03.310831 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 14 10:02:03.310945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:03.310894 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 14 10:02:04.311017 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:04.310978 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 14 10:02:04.311402 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:04.311034 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 14 10:02:05.017283 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:05.017256 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:02:05.311362 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:05.311256 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 14 10:02:05.311362 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:05.311316 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 14 10:02:06.311211 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:06.311165 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 14 10:02:06.311419 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:06.311228 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 14 10:02:07.310250 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:07.310216 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 14 10:02:07.310500 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:07.310280 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 14 10:02:07.615096 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:07.615010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:02:07.716087 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:07.716053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:02:08.314414 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:08.314384 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:02:08.314689 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:08.314665 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:02:08.315458 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:08.315442 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:02:09.921945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:09.921906 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:02:41.746934 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:41.746850 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:02:45.922707 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:02:45.922671 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:03:28.979655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:28.979619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:29.954562 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:29.954526 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:30.062586 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.062540 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g"] Jul 14 10:03:30.066465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.066440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:30.068941 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.068918 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 10:03:30.069097 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.068922 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 10:03:30.073408 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.073381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g"] Jul 14 10:03:30.169635 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.169603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e25ea1a-c70a-4cf3-be8d-471c1c12c71a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g\" (UID: \"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:30.270502 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.270462 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e25ea1a-c70a-4cf3-be8d-471c1c12c71a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g\" (UID: \"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:30.271100 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.271081 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e25ea1a-c70a-4cf3-be8d-471c1c12c71a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g\" (UID: \"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:30.379541 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.379504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:30.524200 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.524176 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g"] Jul 14 10:03:30.526280 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:03:30.526254 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e25ea1a_c70a_4cf3_be8d_471c1c12c71a.slice/crio-06ea0ece95b3ced8639f3fbdf134ce38c6805812613bd47947d62f0db3332d86 WatchSource:0}: Error finding container 06ea0ece95b3ced8639f3fbdf134ce38c6805812613bd47947d62f0db3332d86: Status 404 returned error can't find the container with id 06ea0ece95b3ced8639f3fbdf134ce38c6805812613bd47947d62f0db3332d86 Jul 14 10:03:30.644725 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.644684 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" event={"ID":"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a","Type":"ContainerStarted","Data":"06ea0ece95b3ced8639f3fbdf134ce38c6805812613bd47947d62f0db3332d86"} Jul 14 10:03:30.732983 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.732930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:30.995561 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:30.995478 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:31.650910 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:31.650873 2579 generic.go:356] "Generic (PLEG): container finished" podID="2e25ea1a-c70a-4cf3-be8d-471c1c12c71a" containerID="f8f12c8d7274ea6dd5db546366088c25d8d9d8a42c8383e00c89f33c96df8075" exitCode=0 Jul 14 10:03:31.651309 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:31.650956 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" event={"ID":"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a","Type":"ContainerDied","Data":"f8f12c8d7274ea6dd5db546366088c25d8d9d8a42c8383e00c89f33c96df8075"} Jul 14 10:03:32.797366 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:32.797335 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:32.897275 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:32.897231 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e25ea1a-c70a-4cf3-be8d-471c1c12c71a-maas-api-service-ca\") pod \"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a\" (UID: \"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a\") " Jul 14 10:03:32.897607 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:32.897584 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e25ea1a-c70a-4cf3-be8d-471c1c12c71a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e25ea1a-c70a-4cf3-be8d-471c1c12c71a" (UID: "2e25ea1a-c70a-4cf3-be8d-471c1c12c71a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:03:32.998831 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:32.998719 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e25ea1a-c70a-4cf3-be8d-471c1c12c71a-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:03:33.662096 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:33.662067 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" Jul 14 10:03:33.662275 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:33.662068 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g" event={"ID":"2e25ea1a-c70a-4cf3-be8d-471c1c12c71a","Type":"ContainerDied","Data":"06ea0ece95b3ced8639f3fbdf134ce38c6805812613bd47947d62f0db3332d86"} Jul 14 10:03:33.662275 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:33.662195 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06ea0ece95b3ced8639f3fbdf134ce38c6805812613bd47947d62f0db3332d86" Jul 14 10:03:35.026629 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:35.026589 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:03:40.524794 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:40.524757 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:45.149858 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:45.149827 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:49.725895 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:49.725860 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:03:50.984068 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:50.984031 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:51.050479 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:51.050446 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:55.819707 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:55.819625 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8"] Jul 14 10:03:55.820158 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:55.819883 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" containerID="cri-o://e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a" gracePeriod=30 Jul 14 10:03:55.851267 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:55.851205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:56.066161 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:56.066132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:56.765981 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:56.765949 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:56.868370 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:56.868330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:03:59.129111 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:59.129069 2579 patch_prober.go:28] interesting pod/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 10:03:59.129526 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:03:59.129149 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 10:04:01.083195 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.083168 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:04:01.143051 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.142976 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-certs\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143051 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143012 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-socket\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143051 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143033 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-credential-socket\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143051 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143047 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-data\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143399 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143074 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9a53af0-84a9-45db-bd34-167131532f94-istio-podinfo\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143399 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143140 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9a53af0-84a9-45db-bd34-167131532f94-istiod-ca-cert\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143399 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143179 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-envoy\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143399 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143349 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:01.143399 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143368 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-istio-token\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143656 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143402 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:01.143656 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143465 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzhfx\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-kube-api-access-zzhfx\") pod \"e9a53af0-84a9-45db-bd34-167131532f94\" (UID: \"e9a53af0-84a9-45db-bd34-167131532f94\") " Jul 14 10:04:01.143656 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143573 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a53af0-84a9-45db-bd34-167131532f94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:04:01.143890 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143847 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.143890 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143867 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.143890 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.143882 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9a53af0-84a9-45db-bd34-167131532f94-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.144049 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.144026 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:01.144109 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.144086 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:01.145685 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.145652 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-kube-api-access-zzhfx" (OuterVolumeSpecName: "kube-api-access-zzhfx") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "kube-api-access-zzhfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:04:01.145806 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.145689 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9a53af0-84a9-45db-bd34-167131532f94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:04:01.145899 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.145878 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:01.145954 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.145915 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9a53af0-84a9-45db-bd34-167131532f94" (UID: "e9a53af0-84a9-45db-bd34-167131532f94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:04:01.244964 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.244921 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.244964 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.244954 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.244964 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.244965 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzhfx\" (UniqueName: \"kubernetes.io/projected/e9a53af0-84a9-45db-bd34-167131532f94-kube-api-access-zzhfx\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.244964 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.244974 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.245241 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.244984 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9a53af0-84a9-45db-bd34-167131532f94-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.245241 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.244993 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9a53af0-84a9-45db-bd34-167131532f94-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:01.811737 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.811701 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9a53af0-84a9-45db-bd34-167131532f94" containerID="e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a" exitCode=0 Jul 14 10:04:01.811945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.811765 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" Jul 14 10:04:01.811945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.811779 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" event={"ID":"e9a53af0-84a9-45db-bd34-167131532f94","Type":"ContainerDied","Data":"e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a"} Jul 14 10:04:01.811945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.811836 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8" event={"ID":"e9a53af0-84a9-45db-bd34-167131532f94","Type":"ContainerDied","Data":"b22e81ad7a035a4dec21df00d5fecfc77a0a4a32d31faf395eac296ba694f9ec"} Jul 14 10:04:01.811945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.811860 2579 scope.go:117] "RemoveContainer" containerID="e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a" Jul 14 10:04:01.820951 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.820931 2579 scope.go:117] "RemoveContainer" containerID="633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc" Jul 14 10:04:01.829418 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.829396 2579 scope.go:117] "RemoveContainer" containerID="e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a" Jul 14 10:04:01.829701 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:04:01.829682 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a\": container with ID starting with e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a not found: ID does not exist" containerID="e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a" Jul 14 10:04:01.829900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.829707 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a"} err="failed to get container status \"e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a\": rpc error: code = NotFound desc = could not find container \"e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a\": container with ID starting with e7345062f1ace022088a8798a4ac63e59b30c634848b5396b7e02bc5239f758a not found: ID does not exist" Jul 14 10:04:01.829900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.829727 2579 scope.go:117] "RemoveContainer" containerID="633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc" Jul 14 10:04:01.830369 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:04:01.830349 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc\": container with ID starting with 633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc not found: ID does not exist" containerID="633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc" Jul 14 10:04:01.830420 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.830375 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc"} err="failed to get container status \"633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc\": rpc error: code = NotFound desc = could not find container \"633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc\": container with ID starting with 633fa8d48082d7151ef3a49fd602ff952bc10c6e6e30d976bf8a4ced51b77bfc not found: ID does not exist" Jul 14 10:04:01.831360 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.831340 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8"] Jul 14 10:04:01.837556 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:01.837526 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-06e8a8-openshift-default-5b5bf9c6d5-7jfx8"] Jul 14 10:04:03.637472 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:03.637438 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9a53af0-84a9-45db-bd34-167131532f94" path="/var/lib/kubelet/pods/e9a53af0-84a9-45db-bd34-167131532f94/volumes" Jul 14 10:04:06.681424 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:06.681387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:09.832284 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:09.832248 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:04:11.286127 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:11.286094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:17.184124 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:17.184087 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:17.282229 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:17.282192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:21.937964 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:21.937930 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg"] Jul 14 10:04:21.938453 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:21.938160 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" containerID="cri-o://2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add" gracePeriod=30 Jul 14 10:04:21.959060 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:21.959020 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:22.187298 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.187265 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:22.557189 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.557158 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:22.856622 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.856536 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn"] Jul 14 10:04:22.857000 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.856985 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" Jul 14 10:04:22.857052 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857003 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" Jul 14 10:04:22.857052 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857025 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e25ea1a-c70a-4cf3-be8d-471c1c12c71a" containerName="revoke-keys" Jul 14 10:04:22.857052 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857032 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e25ea1a-c70a-4cf3-be8d-471c1c12c71a" containerName="revoke-keys" Jul 14 10:04:22.857052 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857043 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" Jul 14 10:04:22.857052 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857048 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" Jul 14 10:04:22.857247 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857110 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" Jul 14 10:04:22.857247 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857118 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e25ea1a-c70a-4cf3-be8d-471c1c12c71a" containerName="revoke-keys" Jul 14 10:04:22.857247 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.857126 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a53af0-84a9-45db-bd34-167131532f94" containerName="istio-proxy" Jul 14 10:04:22.860607 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.860581 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.863862 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.863841 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b26ed2-openshift-default-dockercfg-6hrgp\"" Jul 14 10:04:22.873492 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.873460 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn"] Jul 14 10:04:22.882107 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.882079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:22.934249 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934218 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-data\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934452 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934264 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-socket\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934452 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-certs\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934452 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934408 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-envoy\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934452 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934436 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8dc871d-b711-46a3-80c4-75cfd32664c4-istiod-ca-cert\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934624 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934501 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-token\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934624 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptdf\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-kube-api-access-cptdf\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934624 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934584 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-credential-socket\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:22.934624 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:22.934615 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-podinfo\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.035999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.035960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-socket\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.035999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-certs\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036188 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-envoy\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8dc871d-b711-46a3-80c4-75cfd32664c4-istiod-ca-cert\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-token\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036324 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-socket\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036394 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-certs\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptdf\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-kube-api-access-cptdf\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036796 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-credential-socket\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036796 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-podinfo\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036796 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-data\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036956 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036887 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-credential-socket\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036956 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036903 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-data\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.036956 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.036938 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8dc871d-b711-46a3-80c4-75cfd32664c4-istiod-ca-cert\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.038500 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.038484 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-envoy\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.038821 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.038803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-podinfo\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.047887 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.047865 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptdf\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-kube-api-access-cptdf\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.049624 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.049606 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-token\") pod \"e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.171689 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.171602 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:23.235833 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.235352 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:23.317014 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.316974 2579 patch_prober.go:28] interesting pod/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 10:04:23.317171 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.317030 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 10:04:23.334227 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.334202 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn"] Jul 14 10:04:23.336888 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:04:23.336860 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8dc871d_b711_46a3_80c4_75cfd32664c4.slice/crio-2e0a3c6edecc8f75ce640bf072793265c5344e4d642550db578d6f0be907bd83 WatchSource:0}: Error finding container 2e0a3c6edecc8f75ce640bf072793265c5344e4d642550db578d6f0be907bd83: Status 404 returned error can't find the container with id 2e0a3c6edecc8f75ce640bf072793265c5344e4d642550db578d6f0be907bd83 Jul 14 10:04:23.338996 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.338967 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:23.339067 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.339028 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:23.339106 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.339065 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:23.932611 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.932577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" event={"ID":"a8dc871d-b711-46a3-80c4-75cfd32664c4","Type":"ContainerStarted","Data":"29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385"} Jul 14 10:04:23.932822 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.932631 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" event={"ID":"a8dc871d-b711-46a3-80c4-75cfd32664c4","Type":"ContainerStarted","Data":"2e0a3c6edecc8f75ce640bf072793265c5344e4d642550db578d6f0be907bd83"} Jul 14 10:04:23.953083 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:23.953026 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podStartSLOduration=1.95300559 podStartE2EDuration="1.95300559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:04:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:04:23.95269597 +0000 UTC m=+2274.890546440" watchObservedRunningTime="2026-07-14 10:04:23.95300559 +0000 UTC m=+2274.890856060" Jul 14 10:04:24.172006 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:24.171968 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:24.177073 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:24.177047 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:24.938855 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:24.938822 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:24.939846 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:24.939824 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:25.940102 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:25.940063 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:26.112026 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:26.111994 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:26.250309 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:26.250224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:26.468457 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:26.468428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:26.828790 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:26.828759 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:26.928665 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:26.928629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:27.205150 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.205118 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:04:27.277642 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277610 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/626f1d3d-3d95-4963-a121-cc8833f84506-istio-podinfo\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277848 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277692 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-certs\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277848 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277723 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-credential-socket\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277848 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277786 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-socket\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277848 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277828 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-envoy\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277992 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277895 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-istio-token\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277992 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277919 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klwxp\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-kube-api-access-klwxp\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.277992 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.277956 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/626f1d3d-3d95-4963-a121-cc8833f84506-istiod-ca-cert\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.278150 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278030 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:27.278150 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278044 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:27.278150 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278074 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:27.278356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278335 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-data\") pod \"626f1d3d-3d95-4963-a121-cc8833f84506\" (UID: \"626f1d3d-3d95-4963-a121-cc8833f84506\") " Jul 14 10:04:27.278426 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278395 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/626f1d3d-3d95-4963-a121-cc8833f84506-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:04:27.278672 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278654 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.278672 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278672 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.278849 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278682 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.278849 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.278689 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/626f1d3d-3d95-4963-a121-cc8833f84506-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.279093 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.279066 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-data" (OuterVolumeSpecName: "istio-data") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:27.280218 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.280194 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-istio-token" (OuterVolumeSpecName: "istio-token") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:04:27.280308 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.280225 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/626f1d3d-3d95-4963-a121-cc8833f84506-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:04:27.280442 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.280423 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:04:27.280491 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.280457 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-kube-api-access-klwxp" (OuterVolumeSpecName: "kube-api-access-klwxp") pod "626f1d3d-3d95-4963-a121-cc8833f84506" (UID: "626f1d3d-3d95-4963-a121-cc8833f84506"). InnerVolumeSpecName "kube-api-access-klwxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:04:27.379924 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.379833 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.379924 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.379864 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klwxp\" (UniqueName: \"kubernetes.io/projected/626f1d3d-3d95-4963-a121-cc8833f84506-kube-api-access-klwxp\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.379924 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.379878 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.379924 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.379891 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/626f1d3d-3d95-4963-a121-cc8833f84506-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.379924 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.379902 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/626f1d3d-3d95-4963-a121-cc8833f84506-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:04:27.957053 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.956959 2579 generic.go:356] "Generic (PLEG): container finished" podID="626f1d3d-3d95-4963-a121-cc8833f84506" containerID="2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add" exitCode=0 Jul 14 10:04:27.957053 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.957013 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" Jul 14 10:04:27.957053 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.957043 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" event={"ID":"626f1d3d-3d95-4963-a121-cc8833f84506","Type":"ContainerDied","Data":"2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add"} Jul 14 10:04:27.957329 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.957098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg" event={"ID":"626f1d3d-3d95-4963-a121-cc8833f84506","Type":"ContainerDied","Data":"a1f2b85db572cc0230a11011b4dc4b766e3df94a05470cbd054002babb079751"} Jul 14 10:04:27.957329 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.957121 2579 scope.go:117] "RemoveContainer" containerID="2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add" Jul 14 10:04:27.968135 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.968102 2579 scope.go:117] "RemoveContainer" containerID="1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416" Jul 14 10:04:27.977037 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.977012 2579 scope.go:117] "RemoveContainer" containerID="2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add" Jul 14 10:04:27.977339 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:04:27.977318 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add\": container with ID starting with 2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add not found: ID does not exist" containerID="2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add" Jul 14 10:04:27.977423 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.977346 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add"} err="failed to get container status \"2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add\": rpc error: code = NotFound desc = could not find container \"2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add\": container with ID starting with 2454d2a6677aa74abbd16e3d96e0d739aec5dea967cda927f8112a0f55df8add not found: ID does not exist" Jul 14 10:04:27.977423 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.977364 2579 scope.go:117] "RemoveContainer" containerID="1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416" Jul 14 10:04:27.977656 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:04:27.977638 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416\": container with ID starting with 1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416 not found: ID does not exist" containerID="1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416" Jul 14 10:04:27.977718 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.977662 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416"} err="failed to get container status \"1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416\": rpc error: code = NotFound desc = could not find container \"1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416\": container with ID starting with 1e661f606807c159ad6ee7ad8095e4cf621e472eec5240f9afd4e8a035b09416 not found: ID does not exist" Jul 14 10:04:27.978171 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.978152 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg"] Jul 14 10:04:27.981968 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:27.981947 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d7ae40-openshift-default-5bf589f597-q8kxg"] Jul 14 10:04:29.636726 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:29.636691 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" path="/var/lib/kubelet/pods/626f1d3d-3d95-4963-a121-cc8833f84506/volumes" Jul 14 10:04:30.385375 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:30.385342 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:30.846498 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:30.846464 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:31.174791 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.174684 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh"] Jul 14 10:04:31.175316 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.175297 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" Jul 14 10:04:31.175316 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.175318 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" Jul 14 10:04:31.175438 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.175339 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" Jul 14 10:04:31.175438 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.175348 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" Jul 14 10:04:31.175501 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.175452 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" Jul 14 10:04:31.180592 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.180571 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.183516 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.183499 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6cb46d-openshift-default-dockercfg-v8mzs\"" Jul 14 10:04:31.189588 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.189565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh"] Jul 14 10:04:31.203203 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.203172 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:31.315132 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdgw\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-kube-api-access-9mdgw\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315132 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-token\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315377 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d2719d-dba8-4441-8877-ab9a5fc88176-istiod-ca-cert\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315377 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315214 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-socket\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315377 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-credential-socket\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315377 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315299 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-certs\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315377 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315367 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-data\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315609 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315405 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-podinfo\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.315609 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.315443 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-envoy\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416114 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdgw\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-kube-api-access-9mdgw\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416114 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-token\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416368 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d2719d-dba8-4441-8877-ab9a5fc88176-istiod-ca-cert\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416368 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416326 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-socket\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416475 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416428 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-credential-socket\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416538 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-certs\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416538 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-data\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416538 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-podinfo\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416685 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-envoy\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416761 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-socket\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416829 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416765 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-credential-socket\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416829 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416811 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-certs\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.416898 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.416886 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d2719d-dba8-4441-8877-ab9a5fc88176-istiod-ca-cert\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.417192 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.417174 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-data\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.418932 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.418911 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-envoy\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.419052 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.419039 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-podinfo\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.424822 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.424769 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-token\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.424941 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.424924 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdgw\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-kube-api-access-9mdgw\") pod \"e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.491634 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.491599 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:31.637131 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.637108 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh"] Jul 14 10:04:31.639105 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:04:31.639075 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d2719d_dba8_4441_8877_ab9a5fc88176.slice/crio-3b0d8f4c15a1405f3ea447654259912fb9df122a23b04dcc29c4ebfbe8eea9d7 WatchSource:0}: Error finding container 3b0d8f4c15a1405f3ea447654259912fb9df122a23b04dcc29c4ebfbe8eea9d7: Status 404 returned error can't find the container with id 3b0d8f4c15a1405f3ea447654259912fb9df122a23b04dcc29c4ebfbe8eea9d7 Jul 14 10:04:31.641629 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.641595 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:31.641739 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.641678 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:31.641819 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.641762 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:31.819448 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.819413 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:04:31.893818 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.893782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:31.986583 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.986491 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" event={"ID":"30d2719d-dba8-4441-8877-ab9a5fc88176","Type":"ContainerStarted","Data":"3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20"} Jul 14 10:04:31.986583 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:31.986562 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" event={"ID":"30d2719d-dba8-4441-8877-ab9a5fc88176","Type":"ContainerStarted","Data":"3b0d8f4c15a1405f3ea447654259912fb9df122a23b04dcc29c4ebfbe8eea9d7"} Jul 14 10:04:32.025971 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:32.025914 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podStartSLOduration=1.025893358 podStartE2EDuration="1.025893358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:04:32.015031968 +0000 UTC m=+2282.952882448" watchObservedRunningTime="2026-07-14 10:04:32.025893358 +0000 UTC m=+2282.963743832" Jul 14 10:04:32.492730 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:32.492690 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:32.498179 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:32.498152 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:32.992658 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:32.992619 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:32.993652 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:32.993633 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:34.022220 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:34.022183 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:34.117940 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:34.117912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:34.190267 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:34.190237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:34.292972 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:34.292879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:34.796651 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:34.796590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:34.897535 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:34.897504 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:38.483945 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:38.483890 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:40.886374 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:40.886339 2579 patch_prober.go:28] interesting pod/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:51308->10.132.0.50:15021: read: connection reset by peer" start-of-body= Jul 14 10:04:40.886949 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:40.886912 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:51308->10.132.0.50:15021: read: connection reset by peer" Jul 14 10:04:41.036924 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:41.036840 2579 generic.go:356] "Generic (PLEG): container finished" podID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerID="29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385" exitCode=0 Jul 14 10:04:41.037071 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:41.036918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" event={"ID":"a8dc871d-b711-46a3-80c4-75cfd32664c4","Type":"ContainerDied","Data":"29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385"} Jul 14 10:04:41.037336 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:41.037322 2579 scope.go:117] "RemoveContainer" containerID="29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385" Jul 14 10:04:41.038326 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:41.038302 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:41.038379 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:41.038364 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:41.038415 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:41.038394 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:42.043536 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:42.043502 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" event={"ID":"a8dc871d-b711-46a3-80c4-75cfd32664c4","Type":"ContainerStarted","Data":"f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a"} Jul 14 10:04:42.172256 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:42.172216 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:42.172503 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:42.172473 2579 patch_prober.go:28] interesting pod/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:04:42.172722 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:42.172695 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 14 10:04:43.173018 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:43.172976 2579 patch_prober.go:28] interesting pod/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:04:43.173520 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:43.173040 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 14 10:04:44.172835 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:44.172791 2579 patch_prober.go:28] interesting pod/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:04:44.173009 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:44.172858 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 14 10:04:45.172757 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:45.172325 2579 patch_prober.go:28] interesting pod/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:04:45.172757 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:45.172389 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 14 10:04:46.175388 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:46.175360 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:46.175847 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:46.175691 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:46.176314 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:46.176297 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:04:47.207073 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:47.207030 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:47.403024 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:47.402915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:47.993980 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:47.993945 2579 patch_prober.go:28] interesting pod/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 14 10:04:47.994192 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:47.994015 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 14 10:04:48.078558 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:48.078526 2579 generic.go:356] "Generic (PLEG): container finished" podID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerID="3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20" exitCode=0 Jul 14 10:04:48.078729 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:48.078598 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" event={"ID":"30d2719d-dba8-4441-8877-ab9a5fc88176","Type":"ContainerDied","Data":"3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20"} Jul 14 10:04:48.079138 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:48.079125 2579 scope.go:117] "RemoveContainer" containerID="3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20" Jul 14 10:04:48.080120 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:48.080094 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:48.080179 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:48.080165 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:48.080216 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:48.080199 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:04:49.087493 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:49.087456 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" event={"ID":"30d2719d-dba8-4441-8877-ab9a5fc88176","Type":"ContainerStarted","Data":"8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b"} Jul 14 10:04:49.492698 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:49.492598 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:49.492869 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:49.492803 2579 patch_prober.go:28] interesting pod/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 14 10:04:49.492869 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:49.492842 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 14 10:04:49.626249 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:49.626217 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:04:50.493017 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:50.492973 2579 patch_prober.go:28] interesting pod/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 14 10:04:50.493433 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:50.493043 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 14 10:04:51.492403 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:51.492368 2579 patch_prober.go:28] interesting pod/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 14 10:04:51.492577 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:51.492423 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 14 10:04:52.492463 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:52.492430 2579 patch_prober.go:28] interesting pod/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 14 10:04:52.492887 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:52.492502 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 14 10:04:53.497230 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:53.497201 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:53.497698 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:53.497447 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:53.498120 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:53.498100 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:04:55.587769 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:55.587717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:55.688601 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:55.688571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:57.094732 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:57.094693 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:57.193132 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:57.193100 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:04:57.365056 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:04:57.364970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:01.534849 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:01.534818 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:05:01.587195 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:01.587150 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:06.439106 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:06.439068 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:06.539655 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:06.539619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:07.688288 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:07.688243 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:07.937572 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:07.937538 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:12.229732 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:12.229700 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn"] Jul 14 10:05:12.230254 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:12.229969 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" containerID="cri-o://f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a" gracePeriod=30 Jul 14 10:05:12.246584 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:12.246550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:12.468469 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:12.468428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:13.133540 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:13.133499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:13.233163 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:13.233131 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:16.178078 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:16.178045 2579 patch_prober.go:28] interesting pod/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 10:05:16.178438 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:16.178097 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 10:05:17.498775 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.498696 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:05:17.663073 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663039 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-certs\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663073 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663077 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-envoy\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663109 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-socket\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663170 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-data\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663203 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8dc871d-b711-46a3-80c4-75cfd32664c4-istiod-ca-cert\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663234 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-token\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663258 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cptdf\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-kube-api-access-cptdf\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663300 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-podinfo\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663622 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663331 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-credential-socket\") pod \"a8dc871d-b711-46a3-80c4-75cfd32664c4\" (UID: \"a8dc871d-b711-46a3-80c4-75cfd32664c4\") " Jul 14 10:05:17.663622 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663398 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:17.663622 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663474 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:17.663805 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663648 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.663805 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663668 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.663805 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663704 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:17.663805 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.663729 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8dc871d-b711-46a3-80c4-75cfd32664c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:05:17.664224 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.664202 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:17.665367 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.665347 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:17.665719 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.665696 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-kube-api-access-cptdf" (OuterVolumeSpecName: "kube-api-access-cptdf") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "kube-api-access-cptdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:05:17.665881 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.665865 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:05:17.665926 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.665896 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8dc871d-b711-46a3-80c4-75cfd32664c4" (UID: "a8dc871d-b711-46a3-80c4-75cfd32664c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765288 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765321 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8dc871d-b711-46a3-80c4-75cfd32664c4-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765336 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765349 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cptdf\" (UniqueName: \"kubernetes.io/projected/a8dc871d-b711-46a3-80c4-75cfd32664c4-kube-api-access-cptdf\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765366 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765380 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:17.765434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:17.765394 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8dc871d-b711-46a3-80c4-75cfd32664c4-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:18.237504 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.237415 2579 generic.go:356] "Generic (PLEG): container finished" podID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerID="f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a" exitCode=0 Jul 14 10:05:18.237504 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.237480 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" Jul 14 10:05:18.237692 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.237498 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" event={"ID":"a8dc871d-b711-46a3-80c4-75cfd32664c4","Type":"ContainerDied","Data":"f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a"} Jul 14 10:05:18.237692 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.237553 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn" event={"ID":"a8dc871d-b711-46a3-80c4-75cfd32664c4","Type":"ContainerDied","Data":"2e0a3c6edecc8f75ce640bf072793265c5344e4d642550db578d6f0be907bd83"} Jul 14 10:05:18.237692 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.237578 2579 scope.go:117] "RemoveContainer" containerID="f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a" Jul 14 10:05:18.246917 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.246901 2579 scope.go:117] "RemoveContainer" containerID="29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385" Jul 14 10:05:18.254608 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.254590 2579 scope.go:117] "RemoveContainer" containerID="f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a" Jul 14 10:05:18.254871 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:05:18.254852 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a\": container with ID starting with f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a not found: ID does not exist" containerID="f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a" Jul 14 10:05:18.254927 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.254880 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a"} err="failed to get container status \"f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a\": rpc error: code = NotFound desc = could not find container \"f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a\": container with ID starting with f7bef1ac151d5f0aacd82c6f1e3aafc7fb6c796b45a912609f16a9ac9227305a not found: ID does not exist" Jul 14 10:05:18.254927 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.254898 2579 scope.go:117] "RemoveContainer" containerID="29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385" Jul 14 10:05:18.255150 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:05:18.255133 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385\": container with ID starting with 29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385 not found: ID does not exist" containerID="29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385" Jul 14 10:05:18.255194 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.255155 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385"} err="failed to get container status \"29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385\": rpc error: code = NotFound desc = could not find container \"29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385\": container with ID starting with 29b4061e825f28d16152406f9ba491ad86b7888ab52b51721cf5221fca196385 not found: ID does not exist" Jul 14 10:05:18.260054 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.260025 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn"] Jul 14 10:05:18.265246 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:18.265221 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b26ed2-openshift-default-8497f87c6f-sdbnn"] Jul 14 10:05:19.637559 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:19.637523 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" path="/var/lib/kubelet/pods/a8dc871d-b711-46a3-80c4-75cfd32664c4/volumes" Jul 14 10:05:23.079807 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:23.079727 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:26.221286 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:26.221249 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:05:27.704174 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:27.704143 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:33.534794 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:33.534739 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:33.598473 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:33.598443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:38.493202 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:38.493170 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh"] Jul 14 10:05:38.493601 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:38.493377 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" containerID="cri-o://8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b" gracePeriod=30 Jul 14 10:05:38.499542 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:38.499512 2579 patch_prober.go:28] interesting pod/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 10:05:38.499664 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:38.499569 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 10:05:38.519989 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:38.519951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:38.747692 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:38.747554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:39.186542 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.186502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:39.428590 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.428556 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr"] Jul 14 10:05:39.429054 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429034 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" Jul 14 10:05:39.429054 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429056 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" Jul 14 10:05:39.429203 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429083 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" Jul 14 10:05:39.429203 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429091 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" Jul 14 10:05:39.429305 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429223 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="626f1d3d-3d95-4963-a121-cc8833f84506" containerName="istio-proxy" Jul 14 10:05:39.429305 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429236 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" Jul 14 10:05:39.429305 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.429245 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8dc871d-b711-46a3-80c4-75cfd32664c4" containerName="istio-proxy" Jul 14 10:05:39.432351 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.432335 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.435066 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.435046 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-5d32fd-openshift-default-dockercfg-lrz78\"" Jul 14 10:05:39.445060 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.445036 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr"] Jul 14 10:05:39.454118 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.454083 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:39.577769 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.577708 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4shxb\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-kube-api-access-4shxb\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.577769 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.577771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-token\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.577846 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-credential-socket\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.577924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-podinfo\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.577959 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-data\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.577993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6081ee90-cb67-4871-b2fe-7f908a057ebf-istiod-ca-cert\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.578047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-socket\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.578096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-certs\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.578285 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.578135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-envoy\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.584252 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.584219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:39.679562 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679528 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4shxb\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-kube-api-access-4shxb\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679562 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-token\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-credential-socket\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679635 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-podinfo\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-data\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679675 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6081ee90-cb67-4871-b2fe-7f908a057ebf-istiod-ca-cert\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-socket\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-certs\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.679851 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.679817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-envoy\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.680193 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.680138 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-credential-socket\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.680314 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.680200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-socket\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.680314 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.680263 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-certs\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.680424 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.680393 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-data\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.680522 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.680502 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6081ee90-cb67-4871-b2fe-7f908a057ebf-istiod-ca-cert\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.682556 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.682537 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-envoy\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.682700 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.682684 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-podinfo\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.688120 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.688100 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-token\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.688287 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.688267 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4shxb\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-kube-api-access-4shxb\") pod \"e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.742359 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.742272 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:39.886465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.886437 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr"] Jul 14 10:05:39.888392 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:05:39.888363 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6081ee90_cb67_4871_b2fe_7f908a057ebf.slice/crio-96b19cb04dd3e1bd2baf221cf1e0f0e727d63585ed1f57bebccdee3d9bc47c37 WatchSource:0}: Error finding container 96b19cb04dd3e1bd2baf221cf1e0f0e727d63585ed1f57bebccdee3d9bc47c37: Status 404 returned error can't find the container with id 96b19cb04dd3e1bd2baf221cf1e0f0e727d63585ed1f57bebccdee3d9bc47c37 Jul 14 10:05:39.890579 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.890535 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:39.890659 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.890631 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:39.890701 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:39.890677 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:40.355763 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:40.355721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" event={"ID":"6081ee90-cb67-4871-b2fe-7f908a057ebf","Type":"ContainerStarted","Data":"77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7"} Jul 14 10:05:40.355946 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:40.355786 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" event={"ID":"6081ee90-cb67-4871-b2fe-7f908a057ebf","Type":"ContainerStarted","Data":"96b19cb04dd3e1bd2baf221cf1e0f0e727d63585ed1f57bebccdee3d9bc47c37"} Jul 14 10:05:40.376695 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:40.376645 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podStartSLOduration=1.376626353 podStartE2EDuration="1.376626353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:05:40.374422853 +0000 UTC m=+2351.312273324" watchObservedRunningTime="2026-07-14 10:05:40.376626353 +0000 UTC m=+2351.314476823" Jul 14 10:05:40.742864 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:40.742769 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:40.747667 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:40.747645 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:41.361407 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:41.361380 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:41.362605 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:41.362580 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:42.293964 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:42.293923 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:42.382105 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:42.382076 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:42.521738 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:42.521713 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:42.704048 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:42.703959 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:42.971076 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:42.970990 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:43.074654 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.074622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:43.746086 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.746063 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:05:43.820140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820106 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-credential-socket\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820146 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-data\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820370 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820187 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mdgw\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-kube-api-access-9mdgw\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820370 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820228 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-token\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820370 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820283 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-podinfo\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820370 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820327 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-certs\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820534 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820373 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-socket\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820534 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820405 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d2719d-dba8-4441-8877-ab9a5fc88176-istiod-ca-cert\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820534 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820425 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-envoy\") pod \"30d2719d-dba8-4441-8877-ab9a5fc88176\" (UID: \"30d2719d-dba8-4441-8877-ab9a5fc88176\") " Jul 14 10:05:43.820534 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820394 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.820768 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820710 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.820920 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820896 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.820998 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.820933 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.821081 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.821061 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30d2719d-dba8-4441-8877-ab9a5fc88176-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:05:43.821139 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.821113 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-data" (OuterVolumeSpecName: "istio-data") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.822563 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.822519 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:05:43.822670 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.822601 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.822670 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.822638 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-kube-api-access-9mdgw" (OuterVolumeSpecName: "kube-api-access-9mdgw") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "kube-api-access-9mdgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:05:43.822670 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.822658 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-token" (OuterVolumeSpecName: "istio-token") pod "30d2719d-dba8-4441-8877-ab9a5fc88176" (UID: "30d2719d-dba8-4441-8877-ab9a5fc88176"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:05:43.921714 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921681 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.921714 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921707 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.921714 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921715 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d2719d-dba8-4441-8877-ab9a5fc88176-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.921714 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921724 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.922008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921732 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.922008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921739 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mdgw\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-kube-api-access-9mdgw\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.922008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921773 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.922008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:43.921782 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d2719d-dba8-4441-8877-ab9a5fc88176-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:05:44.378416 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.378386 2579 generic.go:356] "Generic (PLEG): container finished" podID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerID="8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b" exitCode=0 Jul 14 10:05:44.378587 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.378433 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" Jul 14 10:05:44.378587 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.378447 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" event={"ID":"30d2719d-dba8-4441-8877-ab9a5fc88176","Type":"ContainerDied","Data":"8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b"} Jul 14 10:05:44.378587 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.378488 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh" event={"ID":"30d2719d-dba8-4441-8877-ab9a5fc88176","Type":"ContainerDied","Data":"3b0d8f4c15a1405f3ea447654259912fb9df122a23b04dcc29c4ebfbe8eea9d7"} Jul 14 10:05:44.378587 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.378511 2579 scope.go:117] "RemoveContainer" containerID="8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b" Jul 14 10:05:44.387688 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.387670 2579 scope.go:117] "RemoveContainer" containerID="3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20" Jul 14 10:05:44.395389 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.395368 2579 scope.go:117] "RemoveContainer" containerID="8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b" Jul 14 10:05:44.395641 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:05:44.395620 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b\": container with ID starting with 8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b not found: ID does not exist" containerID="8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b" Jul 14 10:05:44.395688 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.395651 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b"} err="failed to get container status \"8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b\": rpc error: code = NotFound desc = could not find container \"8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b\": container with ID starting with 8d3bb015e0c48609993b578a76876dc0f9f119143442af37b8cf32aef471513b not found: ID does not exist" Jul 14 10:05:44.395688 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.395669 2579 scope.go:117] "RemoveContainer" containerID="3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20" Jul 14 10:05:44.395948 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:05:44.395931 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20\": container with ID starting with 3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20 not found: ID does not exist" containerID="3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20" Jul 14 10:05:44.395997 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.395955 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20"} err="failed to get container status \"3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20\": rpc error: code = NotFound desc = could not find container \"3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20\": container with ID starting with 3842d3546c6dc250881082c31c4630704ca939546f8ac045cf2e0ae3bdf65e20 not found: ID does not exist" Jul 14 10:05:44.400036 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.400008 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh"] Jul 14 10:05:44.404279 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:44.404259 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6cb46d-openshift-default-587bb6db8c-tz2rh"] Jul 14 10:05:45.636734 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:45.636700 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" path="/var/lib/kubelet/pods/30d2719d-dba8-4441-8877-ab9a5fc88176/volumes" Jul 14 10:05:46.770014 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:46.769981 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:47.105164 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.105072 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:47.366262 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366170 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf"] Jul 14 10:05:47.366654 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366629 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" Jul 14 10:05:47.366654 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366648 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" Jul 14 10:05:47.366824 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366666 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" Jul 14 10:05:47.366824 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366674 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" Jul 14 10:05:47.366824 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366790 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" Jul 14 10:05:47.366824 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.366804 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="30d2719d-dba8-4441-8877-ab9a5fc88176" containerName="istio-proxy" Jul 14 10:05:47.371768 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.371730 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.374117 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.374095 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ad5ead-openshift-default-dockercfg-m8gvp\"" Jul 14 10:05:47.381844 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.381823 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf"] Jul 14 10:05:47.400648 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.400627 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:47.451554 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istiod-ca-cert\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.451733 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451587 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-data\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.451733 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451636 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-token\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.451733 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-socket\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.451733 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-podinfo\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.452000 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-credential-socket\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.452000 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451857 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxbdq\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-kube-api-access-bxbdq\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.452000 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-envoy\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.452000 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.451943 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-certs\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.552935 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.552902 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-data\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.552946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-token\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.552966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-socket\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.552981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-podinfo\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-credential-socket\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxbdq\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-kube-api-access-bxbdq\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-envoy\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553145 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553131 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-certs\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553532 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istiod-ca-cert\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553532 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553367 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-data\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553532 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-socket\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553532 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-credential-socket\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553780 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-certs\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.553862 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.553842 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istiod-ca-cert\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.555421 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.555397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-envoy\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.555800 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.555781 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-podinfo\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.562551 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.562527 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-token\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.565315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.565287 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxbdq\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-kube-api-access-bxbdq\") pod \"e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.628083 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.628002 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:05:47.684654 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.684622 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:47.691428 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.691403 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:47.824330 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.824296 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf"] Jul 14 10:05:47.827724 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:05:47.827696 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda494caa3_e437_4db3_b3e6_4eaa723e48c2.slice/crio-02cafb7f4bd4b0d289c2630b27ead17e5d59d179a2485d3c5684fd3e1b699a8c WatchSource:0}: Error finding container 02cafb7f4bd4b0d289c2630b27ead17e5d59d179a2485d3c5684fd3e1b699a8c: Status 404 returned error can't find the container with id 02cafb7f4bd4b0d289c2630b27ead17e5d59d179a2485d3c5684fd3e1b699a8c Jul 14 10:05:47.829801 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.829770 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:47.829900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.829850 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:47.829969 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:47.829896 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:48.401222 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:48.401192 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" event={"ID":"a494caa3-e437-4db3-b3e6-4eaa723e48c2","Type":"ContainerStarted","Data":"bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61"} Jul 14 10:05:48.401431 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:48.401238 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" event={"ID":"a494caa3-e437-4db3-b3e6-4eaa723e48c2","Type":"ContainerStarted","Data":"02cafb7f4bd4b0d289c2630b27ead17e5d59d179a2485d3c5684fd3e1b699a8c"} Jul 14 10:05:48.427874 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:48.427822 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podStartSLOduration=1.427804581 podStartE2EDuration="1.427804581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:05:48.425679089 +0000 UTC m=+2359.363529571" watchObservedRunningTime="2026-07-14 10:05:48.427804581 +0000 UTC m=+2359.365655050" Jul 14 10:05:48.685372 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:48.685232 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:48.690597 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:48.690568 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:49.406856 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:49.406825 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:49.407725 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:49.407708 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:05:50.357953 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:50.357921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:50.472502 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:50.472402 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:50.554265 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:50.554231 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:50.655510 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:50.655428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:51.117486 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:51.117447 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:51.220245 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:51.220214 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:54.790457 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:54.790422 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:05:56.362693 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.362641 2579 patch_prober.go:28] interesting pod/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:05:56.363272 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.362723 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 14 10:05:56.448355 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.448323 2579 generic.go:356] "Generic (PLEG): container finished" podID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerID="77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7" exitCode=0 Jul 14 10:05:56.448527 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.448393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" event={"ID":"6081ee90-cb67-4871-b2fe-7f908a057ebf","Type":"ContainerDied","Data":"77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7"} Jul 14 10:05:56.448782 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.448767 2579 scope.go:117] "RemoveContainer" containerID="77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7" Jul 14 10:05:56.449725 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.449700 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:56.449798 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.449781 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:56.449832 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:56.449812 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:05:57.455332 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:57.455297 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" event={"ID":"6081ee90-cb67-4871-b2fe-7f908a057ebf","Type":"ContainerStarted","Data":"1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a"} Jul 14 10:05:57.743611 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:57.743517 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:05:57.743801 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:57.743779 2579 patch_prober.go:28] interesting pod/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:05:57.743865 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:57.743827 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 14 10:05:58.743045 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:58.743008 2579 patch_prober.go:28] interesting pod/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:05:58.743435 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:58.743062 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 14 10:05:59.743451 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:59.743353 2579 patch_prober.go:28] interesting pod/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:05:59.743451 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:05:59.743431 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 14 10:06:00.743124 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:00.743082 2579 patch_prober.go:28] interesting pod/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:06:00.743342 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:00.743161 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 14 10:06:01.743574 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:01.743532 2579 patch_prober.go:28] interesting pod/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:06:01.744097 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:01.743613 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 14 10:06:02.746892 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:02.746861 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:06:02.747276 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:02.747205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:06:02.748016 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:02.747998 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:06:03.519895 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.519863 2579 generic.go:356] "Generic (PLEG): container finished" podID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerID="bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61" exitCode=0 Jul 14 10:06:03.520105 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.519937 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" event={"ID":"a494caa3-e437-4db3-b3e6-4eaa723e48c2","Type":"ContainerDied","Data":"bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61"} Jul 14 10:06:03.520434 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.520419 2579 scope.go:117] "RemoveContainer" containerID="bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61" Jul 14 10:06:03.521409 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.521385 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:06:03.521470 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.521451 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:06:03.521516 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.521501 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 10:06:03.730505 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:03.730465 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:06:04.407006 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.406972 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:06:04.526688 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.526656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" event={"ID":"a494caa3-e437-4db3-b3e6-4eaa723e48c2","Type":"ContainerStarted","Data":"a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7"} Jul 14 10:06:04.579647 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.579620 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:06:04.680677 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.680586 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:06:04.684853 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.684816 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:06:04.686161 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.686138 2579 patch_prober.go:28] interesting pod/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:06:04.686288 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:04.686178 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 14 10:06:05.685151 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:05.685112 2579 patch_prober.go:28] interesting pod/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:06:05.685663 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:05.685173 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 14 10:06:06.685827 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:06.685782 2579 patch_prober.go:28] interesting pod/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:06:06.686315 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:06.685857 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 14 10:06:07.295370 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:07.295339 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:06:07.685631 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:07.685535 2579 patch_prober.go:28] interesting pod/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:06:07.685909 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:07.685879 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 14 10:06:08.685732 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:08.685691 2579 patch_prober.go:28] interesting pod/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:06:08.686054 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:08.686022 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 14 10:06:09.688537 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:09.688506 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:06:09.688937 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:09.688846 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:06:09.689489 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:09.689473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:06:11.895673 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:11.895633 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:06:12.012565 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:12.012526 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:06:13.124739 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:13.124703 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:06:29.709535 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:29.709507 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:06:29.716860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:29.716836 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:06:40.363360 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:40.363318 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:06:44.417344 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:06:44.417308 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:07:04.353965 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:07:04.353929 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:07:06.717870 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:07:06.717837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:07:26.021058 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:07:26.021023 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:07:42.919879 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:07:42.919842 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:07:58.215301 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:07:58.215268 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:08:01.933459 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:01.933415 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:08:19.024810 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:19.024773 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:08:34.225107 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:34.225031 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:08:44.638161 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:44.638126 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:08:45.547451 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:45.547419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:08:48.132881 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:48.132847 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:08:48.233523 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:48.233493 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:08:49.934917 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:08:49.934879 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:09:08.187422 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:09:08.187341 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:09:12.021981 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:09:12.021947 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:09:29.774141 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:09:29.774052 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:09:34.324826 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:09:34.324784 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:09:47.009189 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:09:47.009147 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:09:56.422938 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:09:56.422905 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:10:07.197072 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:07.197037 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:07.347206 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:07.347097 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:13.316242 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:13.316209 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:10:15.178823 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:15.178786 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:15.246712 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:15.246684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:18.215448 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:18.215414 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:19.248522 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:19.248489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:26.070283 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:26.070248 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:31.824066 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:31.824026 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:10:49.147295 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:49.147259 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:10:53.320708 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:10:53.320672 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:11:05.329192 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:05.329161 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:05.430123 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:05.430093 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:08.834770 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:08.834712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:11:12.820914 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:12.820882 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:13.055999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:13.055964 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:24.220462 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:24.220380 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:11:29.744257 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:29.744223 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:11:29.750667 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:29.750632 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:11:32.634857 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:32.634807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:33.652519 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:33.652489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:39.013727 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:39.013693 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:11:52.946448 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:52.946408 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7"] Jul 14 10:11:52.950026 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:52.950010 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:11:52.952531 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:52.952510 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 10:11:52.952664 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:52.952594 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m76v\"" Jul 14 10:11:52.957063 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:52.957037 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7"] Jul 14 10:11:53.009257 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:53.009221 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5e16d-02d9-407f-b0a6-ce36284ead6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7\" (UID: \"73a5e16d-02d9-407f-b0a6-ce36284ead6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:11:53.110073 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:53.110038 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5e16d-02d9-407f-b0a6-ce36284ead6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7\" (UID: \"73a5e16d-02d9-407f-b0a6-ce36284ead6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:11:53.110644 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:53.110626 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5e16d-02d9-407f-b0a6-ce36284ead6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7\" (UID: \"73a5e16d-02d9-407f-b0a6-ce36284ead6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:11:53.261525 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:53.261496 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:11:53.405299 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:53.405272 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7"] Jul 14 10:11:53.406952 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:11:53.406926 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a5e16d_02d9_407f_b0a6_ce36284ead6e.slice/crio-2f948ed14f6c90e0b2cc8fa3a1cf50c491118a967f938a74147f7fe1f042ed96 WatchSource:0}: Error finding container 2f948ed14f6c90e0b2cc8fa3a1cf50c491118a967f938a74147f7fe1f042ed96: Status 404 returned error can't find the container with id 2f948ed14f6c90e0b2cc8fa3a1cf50c491118a967f938a74147f7fe1f042ed96 Jul 14 10:11:53.504337 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:53.504266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:54.364100 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:54.364015 2579 generic.go:356] "Generic (PLEG): container finished" podID="73a5e16d-02d9-407f-b0a6-ce36284ead6e" containerID="8b0b3bb49c48b30909a62113c721225ef018d0f974d7cad599fbecc9f980ed0f" exitCode=0 Jul 14 10:11:54.364472 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:54.364097 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" event={"ID":"73a5e16d-02d9-407f-b0a6-ce36284ead6e","Type":"ContainerDied","Data":"8b0b3bb49c48b30909a62113c721225ef018d0f974d7cad599fbecc9f980ed0f"} Jul 14 10:11:54.364472 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:54.364146 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" event={"ID":"73a5e16d-02d9-407f-b0a6-ce36284ead6e","Type":"ContainerStarted","Data":"2f948ed14f6c90e0b2cc8fa3a1cf50c491118a967f938a74147f7fe1f042ed96"} Jul 14 10:11:54.909826 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:54.909793 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:11:55.506093 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:55.506067 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:11:55.634162 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:55.634079 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5e16d-02d9-407f-b0a6-ce36284ead6e-maas-api-service-ca\") pod \"73a5e16d-02d9-407f-b0a6-ce36284ead6e\" (UID: \"73a5e16d-02d9-407f-b0a6-ce36284ead6e\") " Jul 14 10:11:55.634395 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:55.634374 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a5e16d-02d9-407f-b0a6-ce36284ead6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73a5e16d-02d9-407f-b0a6-ce36284ead6e" (UID: "73a5e16d-02d9-407f-b0a6-ce36284ead6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:11:55.735582 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:55.735547 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5e16d-02d9-407f-b0a6-ce36284ead6e-maas-api-service-ca\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:11:56.376379 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:56.376345 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" event={"ID":"73a5e16d-02d9-407f-b0a6-ce36284ead6e","Type":"ContainerDied","Data":"2f948ed14f6c90e0b2cc8fa3a1cf50c491118a967f938a74147f7fe1f042ed96"} Jul 14 10:11:56.376553 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:56.376392 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f948ed14f6c90e0b2cc8fa3a1cf50c491118a967f938a74147f7fe1f042ed96" Jul 14 10:11:56.376553 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:11:56.376359 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7" Jul 14 10:12:03.438808 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:03.438770 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:07.019960 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:07.019923 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:12:08.049649 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:08.049616 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:14.119603 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:14.119567 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:14.181115 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:14.181083 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:18.786828 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:18.786795 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr"] Jul 14 10:12:18.787356 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:18.787022 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" containerID="cri-o://1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a" gracePeriod=30 Jul 14 10:12:18.812413 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:18.812378 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:19.050485 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:19.050392 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:19.885317 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:19.885285 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:19.985708 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:19.985654 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:24.044932 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.044902 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:12:24.093623 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093535 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-data\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093814 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093639 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-podinfo\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093814 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093737 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6081ee90-cb67-4871-b2fe-7f908a057ebf-istiod-ca-cert\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093814 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093793 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-socket\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093852 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4shxb\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-kube-api-access-4shxb\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093895 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-token\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093918 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-envoy\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093965 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-credential-socket\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.093999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.093990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-certs\") pod \"6081ee90-cb67-4871-b2fe-7f908a057ebf\" (UID: \"6081ee90-cb67-4871-b2fe-7f908a057ebf\") " Jul 14 10:12:24.094236 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.094024 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:24.094236 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.094125 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6081ee90-cb67-4871-b2fe-7f908a057ebf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:12:24.094409 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.094361 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:24.095010 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.094431 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:24.095156 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.094536 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-data" (OuterVolumeSpecName: "istio-data") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:24.095277 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.094406 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6081ee90-cb67-4871-b2fe-7f908a057ebf-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.095396 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.095287 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.096350 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.096324 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:24.096545 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.096510 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-kube-api-access-4shxb" (OuterVolumeSpecName: "kube-api-access-4shxb") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "kube-api-access-4shxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:12:24.096650 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.096561 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:12:24.096707 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.096685 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-token" (OuterVolumeSpecName: "istio-token") pod "6081ee90-cb67-4871-b2fe-7f908a057ebf" (UID: "6081ee90-cb67-4871-b2fe-7f908a057ebf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:12:24.195809 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195773 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4shxb\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-kube-api-access-4shxb\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.195809 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195804 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.195809 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195813 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.195809 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195821 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.196084 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195829 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.196084 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195838 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.196084 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.195845 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6081ee90-cb67-4871-b2fe-7f908a057ebf-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:24.528360 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.528321 2579 generic.go:356] "Generic (PLEG): container finished" podID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerID="1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a" exitCode=0 Jul 14 10:12:24.528675 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.528511 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" event={"ID":"6081ee90-cb67-4871-b2fe-7f908a057ebf","Type":"ContainerDied","Data":"1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a"} Jul 14 10:12:24.528675 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.528563 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" event={"ID":"6081ee90-cb67-4871-b2fe-7f908a057ebf","Type":"ContainerDied","Data":"96b19cb04dd3e1bd2baf221cf1e0f0e727d63585ed1f57bebccdee3d9bc47c37"} Jul 14 10:12:24.528675 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.528593 2579 scope.go:117] "RemoveContainer" containerID="1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a" Jul 14 10:12:24.529380 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.529361 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr" Jul 14 10:12:24.539524 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.539505 2579 scope.go:117] "RemoveContainer" containerID="77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7" Jul 14 10:12:24.547231 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.547198 2579 scope.go:117] "RemoveContainer" containerID="1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a" Jul 14 10:12:24.547468 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:12:24.547449 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a\": container with ID starting with 1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a not found: ID does not exist" containerID="1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a" Jul 14 10:12:24.547522 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.547476 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a"} err="failed to get container status \"1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a\": rpc error: code = NotFound desc = could not find container \"1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a\": container with ID starting with 1f76c2e32c74958c7a51badaa44664da20a6ebf64ca14230709a9484782f1f0a not found: ID does not exist" Jul 14 10:12:24.547522 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.547492 2579 scope.go:117] "RemoveContainer" containerID="77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7" Jul 14 10:12:24.547759 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:12:24.547732 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7\": container with ID starting with 77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7 not found: ID does not exist" containerID="77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7" Jul 14 10:12:24.547848 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.547761 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7"} err="failed to get container status \"77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7\": rpc error: code = NotFound desc = could not find container \"77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7\": container with ID starting with 77f590c9e2bad1c3b3e95741cc35e7aff1fec6ca7ec3d83dc482b7c696f6b1f7 not found: ID does not exist" Jul 14 10:12:24.551815 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.551797 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr"] Jul 14 10:12:24.556779 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:24.556760 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-5d32fd-openshift-default-68b85c77dc-lvvqr"] Jul 14 10:12:25.637153 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:25.637121 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" path="/var/lib/kubelet/pods/6081ee90-cb67-4871-b2fe-7f908a057ebf/volumes" Jul 14 10:12:29.681780 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:29.681718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:33.218068 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:33.218033 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4xh6w"] Jul 14 10:12:34.263529 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:34.263493 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:40.409854 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:40.409808 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:40.489198 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:40.489162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:44.946460 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:44.946429 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf"] Jul 14 10:12:44.946947 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:44.946618 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" containerID="cri-o://a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7" gracePeriod=30 Jul 14 10:12:44.965645 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:44.965610 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:45.170789 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:45.170735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:12:50.199946 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.199918 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:12:50.342201 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342172 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istiod-ca-cert\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342201 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342212 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-credential-socket\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342257 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxbdq\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-kube-api-access-bxbdq\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342287 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-token\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342310 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-socket\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342335 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-podinfo\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342360 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-envoy\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342407 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-certs\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342465 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342433 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-data\") pod \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\" (UID: \"a494caa3-e437-4db3-b3e6-4eaa723e48c2\") " Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342508 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342561 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342597 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342804 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istiod-ca-cert\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342810 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342819 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-credential-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.342860 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.342858 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-socket\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.343674 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.343643 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:50.345044 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.345022 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:12:50.345149 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.345123 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:12:50.345149 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.345129 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:12:50.345248 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.345142 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-kube-api-access-bxbdq" (OuterVolumeSpecName: "kube-api-access-bxbdq") pod "a494caa3-e437-4db3-b3e6-4eaa723e48c2" (UID: "a494caa3-e437-4db3-b3e6-4eaa723e48c2"). InnerVolumeSpecName "kube-api-access-bxbdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:12:50.402255 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.402230 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c2cfj_bcbfca45-4486-46da-be18-58947d55bbc6/manager/0.log" Jul 14 10:12:50.444214 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.444183 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxbdq\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-kube-api-access-bxbdq\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.444214 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.444212 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-token\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.444214 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.444221 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-podinfo\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.444446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.444229 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-envoy\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.444446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.444237 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-workload-certs\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.444446 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.444244 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a494caa3-e437-4db3-b3e6-4eaa723e48c2-istio-data\") on node \"ip-10-0-139-42.ec2.internal\" DevicePath \"\"" Jul 14 10:12:50.515008 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.514969 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4qwz7_d21a6b72-b030-427b-a39b-285ac161fdb3/manager/0.log" Jul 14 10:12:50.671397 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.671295 2579 generic.go:356] "Generic (PLEG): container finished" podID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerID="a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7" exitCode=0 Jul 14 10:12:50.671397 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.671365 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" Jul 14 10:12:50.671397 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.671376 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" event={"ID":"a494caa3-e437-4db3-b3e6-4eaa723e48c2","Type":"ContainerDied","Data":"a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7"} Jul 14 10:12:50.671674 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.671450 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf" event={"ID":"a494caa3-e437-4db3-b3e6-4eaa723e48c2","Type":"ContainerDied","Data":"02cafb7f4bd4b0d289c2630b27ead17e5d59d179a2485d3c5684fd3e1b699a8c"} Jul 14 10:12:50.671674 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.671472 2579 scope.go:117] "RemoveContainer" containerID="a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7" Jul 14 10:12:50.680712 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.680694 2579 scope.go:117] "RemoveContainer" containerID="bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61" Jul 14 10:12:50.688433 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.688417 2579 scope.go:117] "RemoveContainer" containerID="a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7" Jul 14 10:12:50.688672 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:12:50.688655 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7\": container with ID starting with a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7 not found: ID does not exist" containerID="a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7" Jul 14 10:12:50.688711 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.688682 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7"} err="failed to get container status \"a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7\": rpc error: code = NotFound desc = could not find container \"a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7\": container with ID starting with a67afa64c0d943f9c2d04630d966d1986110cc0c48b034ac291158127bf460c7 not found: ID does not exist" Jul 14 10:12:50.688711 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.688698 2579 scope.go:117] "RemoveContainer" containerID="bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61" Jul 14 10:12:50.688943 ip-10-0-139-42 kubenswrapper[2579]: E0714 10:12:50.688929 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61\": container with ID starting with bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61 not found: ID does not exist" containerID="bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61" Jul 14 10:12:50.688990 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.688947 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61"} err="failed to get container status \"bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61\": rpc error: code = NotFound desc = could not find container \"bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61\": container with ID starting with bfa54c32e75f33c386e4de4ad90079a297169d95633d9dc74158c2ff56bdaa61 not found: ID does not exist" Jul 14 10:12:50.702086 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.702064 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf"] Jul 14 10:12:50.707195 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.707173 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ad5ead-openshift-default-78fc7bfc5f-4z6xf"] Jul 14 10:12:50.868257 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:50.868231 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pmw4g_46ebccb9-1657-430b-ac2b-245b8ad3cfd1/manager/0.log" Jul 14 10:12:51.213672 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:51.213643 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-vlnbv_ce3867ae-c493-4a1a-8399-1cafabe0ef2d/copy-manifests/0.log" Jul 14 10:12:51.226464 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:51.226428 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-vlnbv_ce3867ae-c493-4a1a-8399-1cafabe0ef2d/manager/0.log" Jul 14 10:12:51.637722 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:51.637682 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" path="/var/lib/kubelet/pods/a494caa3-e437-4db3-b3e6-4eaa723e48c2/volumes" Jul 14 10:12:51.674724 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:51.674695 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733705-rdb9b_bdbb14ca-40af-4357-93e7-82fe51fbfefa/cleanup/0.log" Jul 14 10:12:52.002430 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:52.002348 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-c9216ec8-hnnrw_794bb54e-183b-432b-acaa-b346e9581a50/revoke-keys/1.log" Jul 14 10:12:52.518536 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:52.518505 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-06e8a8-sqz8g_2e25ea1a-c70a-4cf3-be8d-471c1c12c71a/revoke-keys/0.log" Jul 14 10:12:52.760307 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:52.760278 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-40d96159-lwh47_a57b1877-827f-438d-902e-02f58a560d29/revoke-keys/0.log" Jul 14 10:12:52.870625 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:52.870539 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-b2b30ce5-j4wrr_50c51939-4996-4a2b-90c1-604123271f48/revoke-keys/0.log" Jul 14 10:12:52.976714 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:52.976678 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-dc470fea-tbxg7_e761e99b-4616-4f66-b64a-0a0bed2cc613/revoke-keys/0.log" Jul 14 10:12:53.078550 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:53.078519 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-5d32fd-zwnq7_73a5e16d-02d9-407f-b0a6-ce36284ead6e/revoke-keys/0.log" Jul 14 10:12:54.806473 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:54.806444 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dqpnv_a5f18558-cd42-4681-8f58-46c4c08bea7d/manager/0.log" Jul 14 10:12:55.155720 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:55.155686 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hzq2p_df00865d-b84e-4917-9ca9-e4846c081e1f/manager/0.log" Jul 14 10:12:55.266260 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:55.266233 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4xh6w_ca1f5c56-5803-4889-a36f-4190771c3e63/limitador/0.log" Jul 14 10:12:55.376821 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:55.376787 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fdvvx_04458047-5624-4d44-84a0-931854062395/manager/0.log" Jul 14 10:12:55.731687 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:55.731660 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s4n6r_92b5df85-d783-4525-969c-508b1b5612af/istio-proxy/0.log" Jul 14 10:12:55.853955 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:55.853920 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p57p7_a77b754d-9b25-4377-aa46-58ef2c3782c7/discovery/0.log" Jul 14 10:12:56.073547 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:56.073521 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-vwwcc_252393e4-ea9b-4a34-bc5e-e26d1281c146/kube-auth-proxy/0.log" Jul 14 10:12:56.549559 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:12:56.549526 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d68dfd9cb-dkqs8_839ee682-38ac-41bd-9798-cef813d75122/router/0.log" Jul 14 10:13:01.542981 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:01.542951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:13:01.646216 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:01.646185 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:13:04.707070 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:04.707035 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8q6mb_bb6e4a21-f4f3-4033-bc5e-47f28e04b010/global-pull-secret-syncer/0.log" Jul 14 10:13:04.805507 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:04.805479 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" Jul 14 10:13:04.959910 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:04.959829 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-42.ec2.internal_1edd94a745f603ae25b0ce87a74b4849/haproxy/0.log" Jul 14 10:13:09.933345 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:09.933305 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dqpnv_a5f18558-cd42-4681-8f58-46c4c08bea7d/manager/0.log" Jul 14 10:13:10.153063 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:10.153035 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hzq2p_df00865d-b84e-4917-9ca9-e4846c081e1f/manager/0.log" Jul 14 10:13:10.175812 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:10.175783 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4xh6w_ca1f5c56-5803-4889-a36f-4190771c3e63/limitador/0.log" Jul 14 10:13:10.259742 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:10.259713 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fdvvx_04458047-5624-4d44-84a0-931854062395/manager/0.log" Jul 14 10:13:12.065908 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.065881 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-j6dt4_65e9caf5-77ad-46a6-bd56-3f062570093a/cluster-monitoring-operator/0.log" Jul 14 10:13:12.425057 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.424974 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qzlwt_2040c5ca-1d7c-4fd5-a660-764faf71cc0c/node-exporter/0.log" Jul 14 10:13:12.446965 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.446940 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qzlwt_2040c5ca-1d7c-4fd5-a660-764faf71cc0c/kube-rbac-proxy/0.log" Jul 14 10:13:12.469323 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.469299 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qzlwt_2040c5ca-1d7c-4fd5-a660-764faf71cc0c/init-textfile/0.log" Jul 14 10:13:12.589703 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.589672 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/prometheus/0.log" Jul 14 10:13:12.610607 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.610580 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/config-reloader/0.log" Jul 14 10:13:12.635713 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.635688 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/thanos-sidecar/0.log" Jul 14 10:13:12.658338 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.658315 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/kube-rbac-proxy-web/0.log" Jul 14 10:13:12.681367 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.681291 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/kube-rbac-proxy/0.log" Jul 14 10:13:12.708232 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.708207 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/kube-rbac-proxy-thanos/0.log" Jul 14 10:13:12.732355 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:12.732333 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d10a3b2c-8024-4ed2-9b7f-7a9d5c50bda8/init-config-reloader/0.log" Jul 14 10:13:14.258212 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.258173 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-x4gsp_08b281cb-bb74-42c1-94a0-4e8f6e97af98/networking-console-plugin/0.log" Jul 14 10:13:14.901046 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.901017 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/1.log" Jul 14 10:13:14.906420 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.906396 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-ggnwn_277ce53c-c5bb-4e1b-841f-cce8926938a4/console-operator/2.log" Jul 14 10:13:14.989463 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989428 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk"] Jul 14 10:13:14.989824 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989808 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989827 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989843 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989851 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989867 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989872 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989884 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a5e16d-02d9-407f-b0a6-ce36284ead6e" containerName="revoke-keys" Jul 14 10:13:14.989900 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989891 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a5e16d-02d9-407f-b0a6-ce36284ead6e" containerName="revoke-keys" Jul 14 10:13:14.990140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989961 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" Jul 14 10:13:14.990140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989968 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6081ee90-cb67-4871-b2fe-7f908a057ebf" containerName="istio-proxy" Jul 14 10:13:14.990140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989977 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a5e16d-02d9-407f-b0a6-ce36284ead6e" containerName="revoke-keys" Jul 14 10:13:14.990140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989982 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" Jul 14 10:13:14.990140 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.989988 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a494caa3-e437-4db3-b3e6-4eaa723e48c2" containerName="istio-proxy" Jul 14 10:13:14.993360 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.993341 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:14.995728 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.995709 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s29td\"/\"kube-root-ca.crt\"" Jul 14 10:13:14.996475 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.996458 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s29td\"/\"default-dockercfg-zq65n\"" Jul 14 10:13:14.996558 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:14.996461 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s29td\"/\"openshift-service-ca.crt\"" Jul 14 10:13:15.001227 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.001206 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk"] Jul 14 10:13:15.059729 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.059690 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-lib-modules\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.059729 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.059731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-sys\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.059991 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.059823 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwm7v\" (UniqueName: \"kubernetes.io/projected/6ce276c7-e519-40be-a825-73973eec72ec-kube-api-access-mwm7v\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.059991 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.059862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-proc\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.059991 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.059887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-podres\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160537 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-proc\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160537 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160494 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-podres\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160537 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-lib-modules\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160850 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-sys\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160850 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160576 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-proc\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160850 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwm7v\" (UniqueName: \"kubernetes.io/projected/6ce276c7-e519-40be-a825-73973eec72ec-kube-api-access-mwm7v\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160850 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160628 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-sys\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160850 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160635 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-podres\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.160850 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.160661 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ce276c7-e519-40be-a825-73973eec72ec-lib-modules\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.168674 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.168652 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwm7v\" (UniqueName: \"kubernetes.io/projected/6ce276c7-e519-40be-a825-73973eec72ec-kube-api-access-mwm7v\") pod \"perf-node-gather-daemonset-h8zhk\" (UID: \"6ce276c7-e519-40be-a825-73973eec72ec\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.304213 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.304185 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.437891 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.437864 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk"] Jul 14 10:13:15.440148 ip-10-0-139-42 kubenswrapper[2579]: W0714 10:13:15.440111 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6ce276c7_e519_40be_a825_73973eec72ec.slice/crio-e2e28c04a4467c8f0419be6c805fc2a7d627b928c2f230e2e6f26ad346d157ff WatchSource:0}: Error finding container e2e28c04a4467c8f0419be6c805fc2a7d627b928c2f230e2e6f26ad346d157ff: Status 404 returned error can't find the container with id e2e28c04a4467c8f0419be6c805fc2a7d627b928c2f230e2e6f26ad346d157ff Jul 14 10:13:15.661345 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.661315 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-gpwss_d722f4e1-d64e-4e48-ba2e-1e8f006e89ee/download-server/0.log" Jul 14 10:13:15.800522 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.800485 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" event={"ID":"6ce276c7-e519-40be-a825-73973eec72ec","Type":"ContainerStarted","Data":"6a94604ee3c5283739788851881d5cda34e7fc40f01415dbb75eabd091258e9c"} Jul 14 10:13:15.800725 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.800546 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" event={"ID":"6ce276c7-e519-40be-a825-73973eec72ec","Type":"ContainerStarted","Data":"e2e28c04a4467c8f0419be6c805fc2a7d627b928c2f230e2e6f26ad346d157ff"} Jul 14 10:13:15.800725 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.800585 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:15.818104 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:15.818058 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" podStartSLOduration=1.818043039 podStartE2EDuration="1.818043039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:13:15.81613667 +0000 UTC m=+2806.753987141" watchObservedRunningTime="2026-07-14 10:13:15.818043039 +0000 UTC m=+2806.755893508" Jul 14 10:13:16.249934 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:16.249851 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-skbx8_c37a3ab6-cc1c-45cc-a9a1-9bd2d50d51db/volume-data-source-validator/0.log" Jul 14 10:13:17.325334 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:17.325304 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5vbvj_ce209c62-cc38-41b6-8021-d09ba761e072/dns/0.log" Jul 14 10:13:17.348727 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:17.348694 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5vbvj_ce209c62-cc38-41b6-8021-d09ba761e072/kube-rbac-proxy/0.log" Jul 14 10:13:17.522015 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:17.521985 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zxfnl_59d7060b-7f01-4a2f-9e55-503176c06e81/dns-node-resolver/0.log" Jul 14 10:13:18.151201 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:18.151173 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cglm5_fa319f90-fab6-446b-b19f-cf24a7b53761/node-ca/0.log" Jul 14 10:13:19.123784 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:19.123735 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s4n6r_92b5df85-d783-4525-969c-508b1b5612af/istio-proxy/0.log" Jul 14 10:13:19.241056 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:19.241016 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p57p7_a77b754d-9b25-4377-aa46-58ef2c3782c7/discovery/0.log" Jul 14 10:13:19.293232 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:19.293203 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-vwwcc_252393e4-ea9b-4a34-bc5e-e26d1281c146/kube-auth-proxy/0.log" Jul 14 10:13:19.481098 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:19.481020 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d68dfd9cb-dkqs8_839ee682-38ac-41bd-9798-cef813d75122/router/0.log" Jul 14 10:13:20.173000 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:20.172963 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j9jpq_259e96dd-ba4d-4586-8680-ffc219915b18/serve-healthcheck-canary/0.log" Jul 14 10:13:20.678505 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:20.678470 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-tkhdz_a5e16cd1-ac7f-419f-a206-ee417e748911/insights-operator/0.log" Jul 14 10:13:20.679168 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:20.679148 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-tkhdz_a5e16cd1-ac7f-419f-a206-ee417e748911/insights-operator/1.log" Jul 14 10:13:20.842999 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:20.842973 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzx54_cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa/kube-rbac-proxy/0.log" Jul 14 10:13:20.865864 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:20.865837 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzx54_cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa/exporter/0.log" Jul 14 10:13:20.887090 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:20.887063 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzx54_cfad1337-ce60-4d45-b5ba-a94f5a9ea0aa/extractor/0.log" Jul 14 10:13:21.816186 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:21.816151 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h8zhk" Jul 14 10:13:22.968091 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:22.968064 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c2cfj_bcbfca45-4486-46da-be18-58947d55bbc6/manager/0.log" Jul 14 10:13:23.100063 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:23.100031 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4qwz7_d21a6b72-b030-427b-a39b-285ac161fdb3/manager/0.log" Jul 14 10:13:23.288812 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:23.288776 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pmw4g_46ebccb9-1657-430b-ac2b-245b8ad3cfd1/manager/0.log" Jul 14 10:13:23.478684 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:23.478647 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-vlnbv_ce3867ae-c493-4a1a-8399-1cafabe0ef2d/manager/0.log" Jul 14 10:13:23.494602 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:23.494574 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-vlnbv_ce3867ae-c493-4a1a-8399-1cafabe0ef2d/copy-manifests/0.log" Jul 14 10:13:25.478616 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:25.478541 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s4n6r_92b5df85-d783-4525-969c-508b1b5612af/istio-proxy/0.log" Jul 14 10:13:25.588835 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:25.588808 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p57p7_a77b754d-9b25-4377-aa46-58ef2c3782c7/discovery/0.log" Jul 14 10:13:25.621627 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:25.621602 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-vwwcc_252393e4-ea9b-4a34-bc5e-e26d1281c146/kube-auth-proxy/0.log" Jul 14 10:13:25.781970 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:25.781941 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d68dfd9cb-dkqs8_839ee682-38ac-41bd-9798-cef813d75122/router/0.log" Jul 14 10:13:27.080441 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:27.080406 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s4n6r_92b5df85-d783-4525-969c-508b1b5612af/istio-proxy/0.log" Jul 14 10:13:27.190547 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:27.190508 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p57p7_a77b754d-9b25-4377-aa46-58ef2c3782c7/discovery/0.log" Jul 14 10:13:27.224171 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:27.224127 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-vwwcc_252393e4-ea9b-4a34-bc5e-e26d1281c146/kube-auth-proxy/0.log" Jul 14 10:13:27.376955 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:27.376871 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d68dfd9cb-dkqs8_839ee682-38ac-41bd-9798-cef813d75122/router/0.log" Jul 14 10:13:30.529401 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:30.529364 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-sxzmc_a7e096b9-8d20-49c2-b4ca-22041fb503fc/kube-storage-version-migrator-operator/1.log" Jul 14 10:13:30.530250 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:30.530225 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-sxzmc_a7e096b9-8d20-49c2-b4ca-22041fb503fc/kube-storage-version-migrator-operator/0.log" Jul 14 10:13:30.994950 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:30.994917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:13:31.050365 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.050334 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-brzt6_b700b7f3-3775-43aa-941d-b3d90984957c/konnectivity-agent/0.log" limit=1024 Jul 14 10:13:31.523423 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.523389 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2mxt2_de28e022-2681-4080-9ca8-02d15fe6e813/kube-multus/0.log" Jul 14 10:13:31.633039 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.633007 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/kube-multus-additional-cni-plugins/0.log" Jul 14 10:13:31.656219 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.656193 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/egress-router-binary-copy/0.log" Jul 14 10:13:31.678433 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.678402 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/cni-plugins/0.log" Jul 14 10:13:31.702741 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.702715 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/bond-cni-plugin/0.log" Jul 14 10:13:31.726771 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.726722 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/routeoverride-cni/0.log" Jul 14 10:13:31.751576 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.751553 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/whereabouts-cni-bincopy/0.log" Jul 14 10:13:31.780099 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:31.780032 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2g8sm_61a507c5-2f51-4ba3-9866-6e7397b160f4/whereabouts-cni/0.log" Jul 14 10:13:32.211174 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:32.211096 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fpqzv_1bcf77f5-947b-4432-b408-425e3e0612a6/network-metrics-daemon/0.log" Jul 14 10:13:32.232666 ip-10-0-139-42 kubenswrapper[2579]: I0714 10:13:32.232639 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fpqzv_1bcf77f5-947b-4432-b408-425e3e0612a6/kube-rbac-proxy/0.log"