Jul 30 17:24:54.474064 ip-10-0-140-246 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 30 17:24:54.474078 ip-10-0-140-246 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 30 17:24:54.474087 ip-10-0-140-246 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 30 17:24:54.474402 ip-10-0-140-246 systemd[1]: Failed to start Kubernetes Kubelet. Jul 30 17:25:04.639864 ip-10-0-140-246 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 30 17:25:04.639884 ip-10-0-140-246 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b879642bcc5448dd91fbffa696f7819c -- Jul 30 17:26:44.147909 ip-10-0-140-246 systemd[1]: Starting Kubernetes Kubelet... Jul 30 17:26:44.533138 ip-10-0-140-246 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 17:26:44.533138 ip-10-0-140-246 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 17:26:44.533138 ip-10-0-140-246 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 17:26:44.533138 ip-10-0-140-246 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 17:26:44.533138 ip-10-0-140-246 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 17:26:44.535570 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.535490 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541910 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541929 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541932 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541935 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541939 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541941 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541944 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 17:26:44.541937 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541946 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541949 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541952 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541955 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541958 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541960 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541963 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541965 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541968 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541970 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541973 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541976 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541978 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541980 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541983 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541985 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541988 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541990 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541993 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541995 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 17:26:44.542211 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.541997 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542000 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542003 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542005 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542008 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542011 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542015 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542018 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542022 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542024 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542027 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542030 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542033 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542036 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542039 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542043 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542045 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542048 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542051 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542054 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 17:26:44.542690 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542057 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542060 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542062 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542065 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542068 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542070 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542073 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542075 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542078 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542080 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542082 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542085 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542087 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542090 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542092 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542095 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542097 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542099 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542102 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 17:26:44.543170 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542104 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542107 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542109 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542112 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542114 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542116 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542119 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542122 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542125 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542127 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542130 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542135 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542138 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542141 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542144 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542146 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542149 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542152 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542154 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542157 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 17:26:44.543671 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542159 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542161 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542164 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542166 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542169 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542171 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542173 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542176 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542178 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542180 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542183 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542185 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542189 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542902 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542912 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542915 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542918 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542921 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542925 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 17:26:44.544219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542928 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542931 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542934 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542937 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542940 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542943 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542946 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542948 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542951 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542954 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542957 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542960 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542962 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542964 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542967 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542971 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542974 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542977 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542979 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 17:26:44.544694 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542981 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542984 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542986 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542989 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542991 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542994 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542996 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.542998 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543002 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543004 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543007 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543009 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543012 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543015 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543018 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543021 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543024 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543026 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543029 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543031 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 17:26:44.545178 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543034 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543036 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543038 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543041 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543043 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543046 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543049 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543051 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543053 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543056 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543058 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543061 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543063 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543065 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543067 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543070 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543072 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543074 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543079 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 17:26:44.545685 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543083 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543085 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543088 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543091 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543093 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543096 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543100 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543102 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543105 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543107 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543109 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543112 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543114 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543116 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543118 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543121 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543123 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543126 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543128 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543131 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 17:26:44.546149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543133 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543136 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543138 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543141 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543143 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543145 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543148 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543150 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543153 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543155 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543157 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543160 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543162 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543165 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543167 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543544 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543548 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543552 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543555 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543558 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 17:26:44.546655 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543561 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543563 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543566 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543569 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543571 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543574 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543576 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543579 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543581 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543584 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543586 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543608 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543612 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543616 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543620 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543624 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543628 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543632 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543636 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543640 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 17:26:44.547164 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543643 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543646 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543649 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543652 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543654 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543657 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543659 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543662 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543664 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543667 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543670 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543673 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543675 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543679 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543683 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543686 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543689 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543701 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543705 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 17:26:44.547652 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543708 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543711 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543714 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543716 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543719 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543722 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543725 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543727 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543730 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543732 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543735 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543737 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543740 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543742 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543745 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543747 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543750 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543752 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543755 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543758 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 17:26:44.548161 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543760 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543763 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543775 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543780 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543784 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543788 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543792 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543796 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543801 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543823 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543825 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543828 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543830 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543833 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543836 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543842 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543845 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543848 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543851 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543854 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 17:26:44.548706 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543856 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543859 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543862 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543864 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543867 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543869 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543876 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543878 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543881 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543883 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543886 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543888 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543890 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543893 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.543896 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543965 2550 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543972 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543979 2550 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543983 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543987 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543990 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 17:26:44.549190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543994 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.543998 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544001 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544004 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544007 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544010 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544013 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544016 2550 flags.go:64] FLAG: --cgroup-root="" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544019 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544024 2550 flags.go:64] FLAG: --client-ca-file="" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544028 2550 flags.go:64] FLAG: --cloud-provider="external" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544032 2550 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544035 2550 flags.go:64] FLAG: --cluster-domain="" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544038 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544041 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544044 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544047 2550 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544052 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544055 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544058 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544061 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544064 2550 flags.go:64] FLAG: --contention-profiling="false" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544067 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544070 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 17:26:44.549712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544073 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544076 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544079 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544082 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544085 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544088 2550 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544091 2550 flags.go:64] FLAG: --enable-server="true" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544093 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544098 2550 flags.go:64] FLAG: --event-burst="100" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544101 2550 flags.go:64] FLAG: --event-qps="50" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544104 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544107 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544110 2550 flags.go:64] FLAG: --eviction-hard="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544114 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544116 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544119 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544122 2550 flags.go:64] FLAG: --eviction-soft="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544126 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544129 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544132 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544135 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544138 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544141 2550 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544143 2550 flags.go:64] FLAG: --feature-gates="" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544147 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 17:26:44.550274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544153 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544156 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544158 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544161 2550 flags.go:64] FLAG: --healthz-port="10248" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544164 2550 flags.go:64] FLAG: --help="false" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544167 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544170 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544173 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544175 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544179 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544182 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544185 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544188 2550 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544190 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544193 2550 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544196 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544199 2550 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544202 2550 flags.go:64] FLAG: --kube-reserved="" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544204 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544207 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544210 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544212 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544215 2550 flags.go:64] FLAG: --lock-file="" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544218 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 17:26:44.550914 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544222 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544225 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544230 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544234 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544237 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544239 2550 flags.go:64] FLAG: --logging-format="text" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544242 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544245 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544249 2550 flags.go:64] FLAG: --manifest-url="" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544251 2550 flags.go:64] FLAG: --manifest-url-header="" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544256 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544258 2550 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544262 2550 flags.go:64] FLAG: --max-pods="110" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544265 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544268 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544271 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544273 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544276 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544279 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544282 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544289 2550 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544291 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544294 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 17:26:44.551483 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544297 2550 flags.go:64] FLAG: --pod-cidr="" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544300 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544304 2550 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544307 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544310 2550 flags.go:64] FLAG: --pods-per-core="0" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544312 2550 flags.go:64] FLAG: --port="10250" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544315 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544318 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c93aa79e74b3366d" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544321 2550 flags.go:64] FLAG: --qos-reserved="" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544324 2550 flags.go:64] FLAG: --read-only-port="10255" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544327 2550 flags.go:64] FLAG: --register-node="true" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544330 2550 flags.go:64] FLAG: --register-with-taints="" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544334 2550 flags.go:64] FLAG: --registry-burst="10" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544337 2550 flags.go:64] FLAG: --registry-qps="5" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544339 2550 flags.go:64] FLAG: --reserved-cpus="" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544342 2550 flags.go:64] FLAG: --reserved-memory="" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544345 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544349 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544352 2550 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544355 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544358 2550 flags.go:64] FLAG: --runonce="false" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544360 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544363 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544366 2550 flags.go:64] FLAG: --seccomp-default="false" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544369 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 17:26:44.552133 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544371 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544374 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544377 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544380 2550 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544383 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544385 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544388 2550 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544391 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544394 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544397 2550 flags.go:64] FLAG: --system-cgroups="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544399 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544405 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544407 2550 flags.go:64] FLAG: --tls-cert-file="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544410 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544413 2550 flags.go:64] FLAG: --tls-min-version="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544416 2550 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544419 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544422 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544425 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544428 2550 flags.go:64] FLAG: --v="2" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544431 2550 flags.go:64] FLAG: --version="false" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544436 2550 flags.go:64] FLAG: --vmodule="" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544439 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.544443 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544558 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 17:26:44.552831 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544562 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544566 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544570 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544573 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544575 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544578 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544581 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544583 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544586 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544589 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544606 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544610 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544614 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544617 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544619 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544622 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544624 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544627 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544629 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 17:26:44.553432 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544632 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544635 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544637 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544640 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544642 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544645 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544648 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544650 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544653 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544655 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544666 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544670 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544673 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544676 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544679 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544681 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544684 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544686 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544689 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544691 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 17:26:44.553905 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544694 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544696 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544699 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544702 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544704 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544707 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544709 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544712 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544714 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544717 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544719 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544721 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544724 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544726 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544729 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544731 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544734 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544737 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544739 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 17:26:44.554377 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544742 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544744 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544746 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544749 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544752 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544755 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544759 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544761 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544764 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544766 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544769 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544771 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544774 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544776 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544779 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544781 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544783 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544786 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544788 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544791 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 17:26:44.554847 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544793 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544795 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544798 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544800 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544803 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544805 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544808 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544810 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544814 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544816 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544819 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544821 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544823 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544826 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544828 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544831 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544833 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544836 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544840 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 17:26:44.555371 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.544842 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.545554 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.552391 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.552408 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.552426 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.552432 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552486 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552491 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552495 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552497 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552500 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552503 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552506 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552509 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 17:26:44.555900 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552513 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552515 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552518 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552521 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552524 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552527 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552529 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552532 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552534 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552537 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552539 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552542 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552544 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552547 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552549 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552552 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552555 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552557 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552560 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552563 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 17:26:44.556263 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552565 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552569 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552572 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552574 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552577 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552579 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552582 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552584 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552587 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552608 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552612 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552615 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552617 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552620 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552622 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552625 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552628 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552630 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552633 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552635 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 17:26:44.556851 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552638 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552640 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552643 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552646 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552648 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552651 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552653 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552656 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552658 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552663 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552665 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552668 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552672 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552676 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552679 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552681 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552684 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552686 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552689 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 17:26:44.557338 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552691 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552694 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552698 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552701 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552704 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552707 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552709 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552712 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552714 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552717 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552719 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552721 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552724 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552726 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552729 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552731 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552733 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552737 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552741 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 17:26:44.557801 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552744 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552747 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552750 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552752 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552755 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552757 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552761 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552763 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552766 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552768 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552771 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552773 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552776 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.552780 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552884 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 17:26:44.558246 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552888 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552891 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552893 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552909 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552912 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552915 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552917 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552920 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552922 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552924 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552927 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552929 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552931 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552934 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552936 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552938 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552941 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552943 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552946 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552948 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 17:26:44.558639 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552951 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552953 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552956 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552958 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552961 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552963 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552965 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552967 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552970 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552972 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552974 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552977 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552979 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552982 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552984 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552986 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552988 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552991 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552993 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552995 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 17:26:44.559149 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.552997 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553000 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553002 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553005 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553007 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553009 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553012 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553014 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553016 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553019 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553021 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553023 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553027 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553030 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553033 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553036 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553038 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553041 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553044 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 17:26:44.559680 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553047 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553049 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553053 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553055 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553058 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553060 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553062 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553065 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553067 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553069 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553072 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553074 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553077 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553079 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553081 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553083 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553086 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553088 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553090 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553093 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 17:26:44.560175 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553095 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553098 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553100 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553103 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553105 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553107 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553111 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553114 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553117 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553124 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553126 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553129 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553131 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553134 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553136 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553139 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553141 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553143 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 17:26:44.560753 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:44.553146 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.553150 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.553241 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.555135 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.556288 2550 server.go:1019] "Starting client certificate rotation" Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.556385 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.556417 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 17:26:44.561369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.558891 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 17:26:44.562784 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.562762 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 17:26:44.573462 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.573446 2550 log.go:25] "Validated CRI v1 runtime API" Jul 30 17:26:44.576847 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.576830 2550 log.go:25] "Validated CRI v1 image API" Jul 30 17:26:44.577673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.577657 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 17:26:44.579903 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.579884 2550 fs.go:135] Filesystem UUIDs: map[64060603-da77-466c-9ef2-3af10a030957:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8e502c1c-77af-4d24-890d-d6fc21e210da:/dev/nvme0n1p3] Jul 30 17:26:44.579975 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.579902 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 17:26:44.584972 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.584866 2550 manager.go:216] Machine: {Timestamp:2026-07-30 17:26:44.583579462 +0000 UTC m=+0.321311230 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099996 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ee81ebcea48fd9e7337ec70875e5 SystemUUID:ec21ee81-ebce-a48f-d9e7-337ec70875e5 BootID:b879642b-cc54-48dd-91fb-ffa696f7819c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a4:05:37:5c:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a4:05:37:5c:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:7f:44:ce:ea:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 17:26:44.584972 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.584965 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 17:26:44.585097 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.585042 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 17:26:44.585755 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.585740 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 17:26:44.587814 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.587789 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 17:26:44.587947 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.587817 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-246.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 17:26:44.588973 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.588963 2550 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 17:26:44.589004 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.588976 2550 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 17:26:44.589004 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.588983 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 17:26:44.589650 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.589587 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 17:26:44.590487 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.590478 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 17:26:44.592102 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592092 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 30 17:26:44.592208 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592198 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 17:26:44.592249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592227 2550 kubelet.go:486] "Attempting to sync node with API server" Jul 30 17:26:44.592249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592242 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 17:26:44.592316 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592258 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 17:26:44.592316 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592266 2550 kubelet.go:398] "Adding apiserver pod source" Jul 30 17:26:44.592316 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592276 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 17:26:44.592628 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592615 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 17:26:44.592684 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.592659 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 17:26:44.595098 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.595082 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 17:26:44.596638 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.596623 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 17:26:44.596706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.596654 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 17:26:44.597969 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.597956 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.597973 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.597979 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.597985 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.597993 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.597998 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598005 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598010 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598017 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598022 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598030 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 17:26:44.598030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598035 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 17:26:44.598787 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598774 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 17:26:44.598787 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.598784 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 17:26:44.600616 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.600577 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-246.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 17:26:44.600702 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.600625 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-246.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 17:26:44.600702 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.600650 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 17:26:44.601987 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.601976 2550 server.go:1268] "Started kubelet" Jul 30 17:26:44.602627 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.602408 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 17:26:44.602877 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.602819 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 17:26:44.602952 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.602902 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 17:26:44.603332 ip-10-0-140-246 systemd[1]: Started Kubernetes Kubelet. Jul 30 17:26:44.605165 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.605142 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 17:26:44.605719 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.605699 2550 server.go:310] "Adding debug handlers to kubelet server" Jul 30 17:26:44.606550 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.606530 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 17:26:44.611497 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.611483 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 17:26:44.611586 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.611535 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 17:26:44.611586 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.611563 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:44.611858 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.610795 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-246.ec2.internal.18c7214ff6c55115 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-246.ec2.internal,UID:ip-10-0-140-246.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-246.ec2.internal,},FirstTimestamp:2026-07-30 17:26:44.601950485 +0000 UTC m=+0.339682253,LastTimestamp:2026-07-30 17:26:44.601950485 +0000 UTC m=+0.339682253,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-246.ec2.internal,}" Jul 30 17:26:44.612004 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.611986 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 17:26:44.612004 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.611988 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 17:26:44.612157 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612014 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 17:26:44.612157 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612117 2550 reconstruct.go:97] "Volume reconstruction finished" Jul 30 17:26:44.612157 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612123 2550 reconciler.go:29] "Reconciler: start to sync state" Jul 30 17:26:44.612387 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612373 2550 factory.go:55] Registering systemd factory Jul 30 17:26:44.612443 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612389 2550 factory.go:223] Registration of the systemd container factory successfully Jul 30 17:26:44.612740 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612721 2550 factory.go:153] Registering CRI-O factory Jul 30 17:26:44.612740 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612742 2550 factory.go:223] Registration of the crio container factory successfully Jul 30 17:26:44.612881 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612799 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 17:26:44.612881 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612823 2550 factory.go:103] Registering Raw factory Jul 30 17:26:44.612881 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.612839 2550 manager.go:1196] Started watching for new ooms in manager Jul 30 17:26:44.613456 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.613432 2550 manager.go:349] Starting recovery of all containers Jul 30 17:26:44.624058 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.623927 2550 manager.go:354] Recovery completed Jul 30 17:26:44.624209 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.624187 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 17:26:44.624303 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.624211 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-246.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 17:26:44.628131 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.628117 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.629871 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.629852 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.629931 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.629890 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.629931 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.629906 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.630401 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.630385 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 17:26:44.630401 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.630400 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 17:26:44.630544 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.630409 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.630544 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.630416 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 30 17:26:44.631037 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.631022 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.631094 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.631050 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.631094 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.631059 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.631972 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.631959 2550 policy_none.go:49] "None policy: Start" Jul 30 17:26:44.632030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.631978 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 17:26:44.632030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.631991 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 17:26:44.632962 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.632950 2550 policy_none.go:47] "Start" Jul 30 17:26:44.634765 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.634750 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qscwz" Jul 30 17:26:44.640926 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.640812 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-246.ec2.internal.18c7214ff86f5f60 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-246.ec2.internal,UID:ip-10-0-140-246.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-246.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-246.ec2.internal,},FirstTimestamp:2026-07-30 17:26:44.62987248 +0000 UTC m=+0.367604256,LastTimestamp:2026-07-30 17:26:44.62987248 +0000 UTC m=+0.367604256,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-246.ec2.internal,}" Jul 30 17:26:44.642984 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.642929 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-246.ec2.internal.18c7214ff86fc235 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-246.ec2.internal,UID:ip-10-0-140-246.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-246.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-246.ec2.internal,},FirstTimestamp:2026-07-30 17:26:44.629897781 +0000 UTC m=+0.367629554,LastTimestamp:2026-07-30 17:26:44.629897781 +0000 UTC m=+0.367629554,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-246.ec2.internal,}" Jul 30 17:26:44.643062 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.642996 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qscwz" Jul 30 17:26:44.667682 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.667669 2550 manager.go:337] "Starting Device Plugin manager" Jul 30 17:26:44.667744 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.667698 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 17:26:44.667744 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.667708 2550 server.go:87] "Starting device plugin registration server" Jul 30 17:26:44.667970 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.667956 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 17:26:44.668023 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.667971 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 17:26:44.668070 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.668042 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 17:26:44.668129 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.668115 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 17:26:44.668180 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.668131 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 17:26:44.668625 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.668588 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 17:26:44.668685 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.668656 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:44.768306 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.768284 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.769327 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.769312 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.769384 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.769342 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.769384 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.769352 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.769446 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.769392 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.778447 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.778429 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.778526 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.778449 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-246.ec2.internal\": node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:44.795115 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.795069 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 17:26:44.796388 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.796374 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 17:26:44.796440 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.796410 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 17:26:44.796440 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.796427 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 17:26:44.796523 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.796455 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 17:26:44.800851 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.800818 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 17:26:44.802348 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.802333 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:44.896997 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.896959 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal"] Jul 30 17:26:44.897086 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.897039 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.897981 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.897956 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.898080 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.897991 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.898080 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.898002 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.899363 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.899351 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.899478 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.899464 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.899516 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.899506 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.900134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.900115 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.900134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.900125 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.900258 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.900144 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.900258 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.900144 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.900258 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.900154 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.900258 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.900162 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.901465 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.901450 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.901549 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.901474 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:44.902082 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.902065 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:44.902150 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.902095 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:44.902150 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.902110 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:44.903125 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.903109 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:44.914507 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.914490 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c20407b4053ae7e9ce65385b9e6d586a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal\" (UID: \"c20407b4053ae7e9ce65385b9e6d586a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.914583 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.914513 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2288bbe26fd7af2100c8809ffb7ffbce-config\") pod \"kube-apiserver-proxy-ip-10-0-140-246.ec2.internal\" (UID: \"2288bbe26fd7af2100c8809ffb7ffbce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.914583 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:44.914531 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c20407b4053ae7e9ce65385b9e6d586a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal\" (UID: \"c20407b4053ae7e9ce65385b9e6d586a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.928154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.928137 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-246.ec2.internal\" not found" node="ip-10-0-140-246.ec2.internal" Jul 30 17:26:44.932020 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:44.932003 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-246.ec2.internal\" not found" node="ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.003696 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.003681 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.015120 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.015100 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c20407b4053ae7e9ce65385b9e6d586a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal\" (UID: \"c20407b4053ae7e9ce65385b9e6d586a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.015188 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.015123 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c20407b4053ae7e9ce65385b9e6d586a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal\" (UID: \"c20407b4053ae7e9ce65385b9e6d586a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.015188 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.015141 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2288bbe26fd7af2100c8809ffb7ffbce-config\") pod \"kube-apiserver-proxy-ip-10-0-140-246.ec2.internal\" (UID: \"2288bbe26fd7af2100c8809ffb7ffbce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.015188 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.015180 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2288bbe26fd7af2100c8809ffb7ffbce-config\") pod \"kube-apiserver-proxy-ip-10-0-140-246.ec2.internal\" (UID: \"2288bbe26fd7af2100c8809ffb7ffbce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.015306 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.015223 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c20407b4053ae7e9ce65385b9e6d586a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal\" (UID: \"c20407b4053ae7e9ce65385b9e6d586a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.015306 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.015221 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c20407b4053ae7e9ce65385b9e6d586a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal\" (UID: \"c20407b4053ae7e9ce65385b9e6d586a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.104557 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.104491 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.204976 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.204946 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.231393 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.231373 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.235566 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.235361 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.305732 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.305703 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.406309 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.406253 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.506729 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.506701 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.545857 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.545830 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 17:26:45.555866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.555837 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 17:26:45.555968 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.555951 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 17:26:45.556025 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.556003 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 17:26:45.607247 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.607225 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.647578 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.647544 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 17:21:45 +0000 UTC" deadline="2028-02-21 07:32:39.621799836 +0000 UTC" Jul 30 17:26:45.647578 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.647574 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13694h5m53.974229774s" Jul 30 17:26:45.664751 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.664702 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 17:26:45.665581 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.665563 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientMemory" Jul 30 17:26:45.665689 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.665613 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 17:26:45.665689 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.665628 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeHasSufficientPID" Jul 30 17:26:45.708305 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.708277 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.779401 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:45.779363 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2288bbe26fd7af2100c8809ffb7ffbce.slice/crio-73a948d71e2cdfc2420b091a135ca99e3513891385cc8ad78514d1e550953576 WatchSource:0}: Error finding container 73a948d71e2cdfc2420b091a135ca99e3513891385cc8ad78514d1e550953576: Status 404 returned error can't find the container with id 73a948d71e2cdfc2420b091a135ca99e3513891385cc8ad78514d1e550953576 Jul 30 17:26:45.780268 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:45.780235 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20407b4053ae7e9ce65385b9e6d586a.slice/crio-b9eaf5f3edde61973e75299fbcab23573ee5496704c7aa9dc671923522723ad7 WatchSource:0}: Error finding container b9eaf5f3edde61973e75299fbcab23573ee5496704c7aa9dc671923522723ad7: Status 404 returned error can't find the container with id b9eaf5f3edde61973e75299fbcab23573ee5496704c7aa9dc671923522723ad7 Jul 30 17:26:45.785184 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.785166 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 17:26:45.790073 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.789849 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 17:26:45.800345 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.800306 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" event={"ID":"2288bbe26fd7af2100c8809ffb7ffbce","Type":"ContainerStarted","Data":"73a948d71e2cdfc2420b091a135ca99e3513891385cc8ad78514d1e550953576"} Jul 30 17:26:45.801700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.801680 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" event={"ID":"c20407b4053ae7e9ce65385b9e6d586a","Type":"ContainerStarted","Data":"b9eaf5f3edde61973e75299fbcab23573ee5496704c7aa9dc671923522723ad7"} Jul 30 17:26:45.808842 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:45.808819 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-246.ec2.internal\" not found" Jul 30 17:26:45.823245 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.823223 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 17:26:45.912573 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.912552 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.923171 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.923132 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 17:26:45.924523 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.924511 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" Jul 30 17:26:45.937089 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:45.937074 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 17:26:47.599436 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.599410 2550 apiserver.go:52] "Watching apiserver" Jul 30 17:26:47.608210 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.608186 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 17:26:47.609993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.609966 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rxxz2","openshift-dns/node-resolver-mptvx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal","openshift-multus/multus-additional-cni-plugins-x7pxk","openshift-multus/network-metrics-daemon-8nlrg","openshift-network-diagnostics/network-check-target-9b5ft","openshift-network-operator/iptables-alerter-4xv95","kube-system/global-pull-secret-syncer-8lrfj","kube-system/konnectivity-agent-wg4tv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4","openshift-image-registry/node-ca-8hj5m","openshift-multus/multus-8ktvv","openshift-ovn-kubernetes/ovnkube-node-m8qn9"] Jul 30 17:26:47.611454 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.611435 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.612677 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.612659 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.614129 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.614107 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-92m2b\"" Jul 30 17:26:47.614233 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.614155 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 17:26:47.614233 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.614219 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 17:26:47.615019 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.614997 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.615127 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.615019 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.615466 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.615450 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 17:26:47.615557 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.615502 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5tj5c\"" Jul 30 17:26:47.615557 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.615530 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.616457 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.616440 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.618134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.617457 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 17:26:47.618134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.617513 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:47.618134 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.617629 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:26:47.618134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.617673 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q5r7k\"" Jul 30 17:26:47.618134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.617824 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 17:26:47.618134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.617946 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 17:26:47.618897 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.618875 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 17:26:47.618999 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.618924 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.618999 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.618949 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.619346 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.619327 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-25b2d\"" Jul 30 17:26:47.619581 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.619565 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.619665 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.619612 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:47.619844 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.619795 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:47.620620 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.620264 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.620786 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.620413 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 17:26:47.621165 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.620483 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 17:26:47.623836 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.623169 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.625695 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.625673 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.626313 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.626294 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 17:26:47.626409 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.626313 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.626472 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.626299 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x625v\"" Jul 30 17:26:47.626989 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.626968 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 17:26:47.627152 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627134 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-log-socket\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627237 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627162 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovnkube-config\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627237 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627178 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-node-log\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627237 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627197 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-cni-bin\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627237 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627233 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-kubelet\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627259 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-run-netns\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627280 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-systemd\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627313 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627327 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627327 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-var-lib-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627398 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627423 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-registration-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627446 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-host-slash\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627468 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn5k7\" (UniqueName: \"kubernetes.io/projected/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-kube-api-access-fn5k7\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627490 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627522 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-etc-selinux\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627537 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv9j2\" (UniqueName: \"kubernetes.io/projected/d8d14018-07a9-454d-9efd-87d7e2c424fd-kube-api-access-sv9j2\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627563 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-env-overrides\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627578 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovnkube-script-lib\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627594 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627607 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-run-ovn-kubernetes\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.627732 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627672 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627921 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-cni-netd\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627946 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-systemd-units\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627970 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-slash\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.627991 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-iptables-alerter-script\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628015 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfd87863-1f53-4966-ba91-7f88ea173688-agent-certs\") pod \"konnectivity-agent-wg4tv\" (UID: \"bfd87863-1f53-4966-ba91-7f88ea173688\") " pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628033 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-device-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628046 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-ovn\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628059 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628076 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovn-node-metrics-cert\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628117 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfd87863-1f53-4966-ba91-7f88ea173688-konnectivity-ca\") pod \"konnectivity-agent-wg4tv\" (UID: \"bfd87863-1f53-4966-ba91-7f88ea173688\") " pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628148 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-socket-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628175 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-sys-fs\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.628204 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628207 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-etc-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628845 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628230 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhk7c\" (UniqueName: \"kubernetes.io/projected/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-kube-api-access-mhk7c\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.628845 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628619 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t7dkp\"" Jul 30 17:26:47.628845 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628645 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 17:26:47.628990 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.628896 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.629123 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.629095 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:47.630177 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.630155 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.630362 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.630341 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.630438 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.630368 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.630438 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.630430 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-76sws\"" Jul 30 17:26:47.631635 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.631617 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 17:26:47.631715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.631678 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.633040 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.633001 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.633139 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.633040 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pnp6q\"" Jul 30 17:26:47.633139 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.633090 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.633293 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.633277 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 17:26:47.634502 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.634486 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 17:26:47.634781 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.634763 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 17:26:47.634883 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.634857 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nkmvt\"" Jul 30 17:26:47.651723 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.651703 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-52rtg" Jul 30 17:26:47.659772 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.659752 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-52rtg" Jul 30 17:26:47.718710 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.718687 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 17:26:47.728514 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728489 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-run-ovn-kubernetes\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.728619 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728522 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-cni-netd\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.728619 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728545 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-slash\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.728619 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728567 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-iptables-alerter-script\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.728619 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728593 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-os-release\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.728619 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728614 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-kubelet\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728634 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-etc-kubernetes\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728657 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728682 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfd87863-1f53-4966-ba91-7f88ea173688-agent-certs\") pod \"konnectivity-agent-wg4tv\" (UID: \"bfd87863-1f53-4966-ba91-7f88ea173688\") " pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728706 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-device-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728728 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-ovn\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728749 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-cnibin\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728772 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-cni-multus\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728794 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-system-cni-dir\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728836 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-cnibin\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728858 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-kubernetes\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.728895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728888 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfd87863-1f53-4966-ba91-7f88ea173688-konnectivity-ca\") pod \"konnectivity-agent-wg4tv\" (UID: \"bfd87863-1f53-4966-ba91-7f88ea173688\") " pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728920 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-etc-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728943 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-multus-certs\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728967 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6vsb\" (UniqueName: \"kubernetes.io/projected/58726591-2751-4c7d-88c4-b916a4cdc736-kube-api-access-s6vsb\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.728998 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp275\" (UniqueName: \"kubernetes.io/projected/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-kube-api-access-zp275\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729038 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ceb77fd3-c224-4c26-aad4-923719f14703-host\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729061 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-log-socket\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729082 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-conf-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729106 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rm2g\" (UniqueName: \"kubernetes.io/projected/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-kube-api-access-7rm2g\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729130 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysctl-d\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729153 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-node-log\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729176 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-kubelet\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729199 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-run-netns\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729221 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729248 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-host-slash\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729271 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-lib-modules\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729294 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0107a065-c0b7-496d-ad29-8d02b454374c-hosts-file\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.729432 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729321 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-registration-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729346 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-netns\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729374 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-cni-bin\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729399 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75rpk\" (UniqueName: \"kubernetes.io/projected/7f903377-3e78-436b-8d95-4be29aed46eb-kube-api-access-75rpk\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729422 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-sys\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729442 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-tuned\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729469 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-tmp\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729493 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-k8s-cni-cncf-io\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729517 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/246d90ee-d830-4bf0-b942-ffbfa32a98f9-kubelet-config\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729547 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729569 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-run\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729593 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b4zc\" (UniqueName: \"kubernetes.io/projected/ceb77fd3-c224-4c26-aad4-923719f14703-kube-api-access-8b4zc\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729618 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9x5n\" (UniqueName: \"kubernetes.io/projected/0107a065-c0b7-496d-ad29-8d02b454374c-kube-api-access-r9x5n\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729644 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729669 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv9j2\" (UniqueName: \"kubernetes.io/projected/d8d14018-07a9-454d-9efd-87d7e2c424fd-kube-api-access-sv9j2\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729702 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-env-overrides\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730281 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729728 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovnkube-script-lib\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729753 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729776 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-var-lib-kubelet\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729798 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-host\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729841 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0107a065-c0b7-496d-ad29-8d02b454374c-tmp-dir\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729866 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-systemd-units\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729891 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729919 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729945 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovn-node-metrics-cert\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.729969 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-system-cni-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730020 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730048 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-socket-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730075 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-sys-fs\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730113 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhk7c\" (UniqueName: \"kubernetes.io/projected/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-kube-api-access-mhk7c\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730138 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovnkube-config\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730181 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-cni-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730206 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-hostroot\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.730966 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730231 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/246d90ee-d830-4bf0-b942-ffbfa32a98f9-dbus\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730254 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ceb77fd3-c224-4c26-aad4-923719f14703-serviceca\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730284 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-cni-bin\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730308 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-systemd\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730333 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-var-lib-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730362 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-cni-binary-copy\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730388 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730414 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-os-release\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730441 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730466 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-modprobe-d\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730489 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysctl-conf\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730514 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-systemd\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730540 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn5k7\" (UniqueName: \"kubernetes.io/projected/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-kube-api-access-fn5k7\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730565 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-socket-dir-parent\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730590 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-daemon-config\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730615 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysconfig\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-etc-selinux\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.731706 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730744 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-log-socket\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730847 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-device-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730738 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-etc-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730875 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-cni-netd\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730916 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-run-ovn-kubernetes\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730923 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-ovn\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730913 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730943 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-host-slash\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730955 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-kubelet\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730969 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-registration-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.730986 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-node-log\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731001 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-slash\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731036 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-run-netns\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731050 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-cni-bin\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731103 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-var-lib-openvswitch\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731135 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-run-systemd\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731199 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-sys-fs\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731249 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-socket-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.732504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731269 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731459 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfd87863-1f53-4966-ba91-7f88ea173688-konnectivity-ca\") pod \"konnectivity-agent-wg4tv\" (UID: \"bfd87863-1f53-4966-ba91-7f88ea173688\") " pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731540 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-env-overrides\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731627 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731682 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-etc-selinux\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731706 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-iptables-alerter-script\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731771 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-systemd-units\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.731870 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovnkube-config\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.732142 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovnkube-script-lib\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.733337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.732224 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d8d14018-07a9-454d-9efd-87d7e2c424fd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.735120 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.735099 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-ovn-node-metrics-cert\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.735227 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.735209 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfd87863-1f53-4966-ba91-7f88ea173688-agent-certs\") pod \"konnectivity-agent-wg4tv\" (UID: \"bfd87863-1f53-4966-ba91-7f88ea173688\") " pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.741937 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.741915 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:26:47.742033 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.741945 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:26:47.742033 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.741958 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:47.742184 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.742045 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:48.242007546 +0000 UTC m=+2.974522213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:47.744468 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.744413 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv9j2\" (UniqueName: \"kubernetes.io/projected/d8d14018-07a9-454d-9efd-87d7e2c424fd-kube-api-access-sv9j2\") pod \"aws-ebs-csi-driver-node-qf6p4\" (UID: \"d8d14018-07a9-454d-9efd-87d7e2c424fd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.744468 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.744451 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn5k7\" (UniqueName: \"kubernetes.io/projected/0d5844c4-84f8-41d2-b3d8-edfd1184fb22-kube-api-access-fn5k7\") pod \"iptables-alerter-4xv95\" (UID: \"0d5844c4-84f8-41d2-b3d8-edfd1184fb22\") " pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.744854 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.744835 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhk7c\" (UniqueName: \"kubernetes.io/projected/8d9a6c7b-dbdd-4a56-871d-27aaede78c97-kube-api-access-mhk7c\") pod \"ovnkube-node-m8qn9\" (UID: \"8d9a6c7b-dbdd-4a56-871d-27aaede78c97\") " pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.831299 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831224 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-os-release\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.831299 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831265 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:47.831299 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831289 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-modprobe-d\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831312 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysctl-conf\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831336 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-systemd\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831359 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-socket-dir-parent\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831361 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-os-release\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831383 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-daemon-config\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831403 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysconfig\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831431 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-os-release\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831449 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-kubelet\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831469 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-etc-kubernetes\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831492 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831497 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysctl-conf\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831516 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-cnibin\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.831533 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831537 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-cni-multus\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831552 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-system-cni-dir\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831571 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-cnibin\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.831581 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831592 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-kubernetes\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831611 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-multus-certs\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.831638 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:48.331620155 +0000 UTC m=+3.064134830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831669 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-multus-certs\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831678 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6vsb\" (UniqueName: \"kubernetes.io/projected/58726591-2751-4c7d-88c4-b916a4cdc736-kube-api-access-s6vsb\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831708 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-systemd\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831762 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-socket-dir-parent\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832201 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831837 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-modprobe-d\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.832715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832345 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-cnibin\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832408 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysconfig\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.832715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832470 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-os-release\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832511 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-kubelet\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832551 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-etc-kubernetes\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832777 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-daemon-config\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832843 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-cni-multus\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832882 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-system-cni-dir\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832909 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-cnibin\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832940 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-kubernetes\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.831716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp275\" (UniqueName: \"kubernetes.io/projected/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-kube-api-access-zp275\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.832993 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.832967 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ceb77fd3-c224-4c26-aad4-923719f14703-host\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833004 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-conf-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833026 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rm2g\" (UniqueName: \"kubernetes.io/projected/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-kube-api-access-7rm2g\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833070 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysctl-d\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833097 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833109 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-lib-modules\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833131 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0107a065-c0b7-496d-ad29-8d02b454374c-hosts-file\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833155 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-netns\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833185 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-cni-bin\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833208 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75rpk\" (UniqueName: \"kubernetes.io/projected/7f903377-3e78-436b-8d95-4be29aed46eb-kube-api-access-75rpk\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833232 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-sys\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833261 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-tuned\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833284 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-tmp\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833305 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-k8s-cni-cncf-io\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833331 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/246d90ee-d830-4bf0-b942-ffbfa32a98f9-kubelet-config\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833354 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.833372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833355 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ceb77fd3-c224-4c26-aad4-923719f14703-host\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833377 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-run\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833395 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-conf-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833413 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b4zc\" (UniqueName: \"kubernetes.io/projected/ceb77fd3-c224-4c26-aad4-923719f14703-kube-api-access-8b4zc\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833438 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9x5n\" (UniqueName: \"kubernetes.io/projected/0107a065-c0b7-496d-ad29-8d02b454374c-kube-api-access-r9x5n\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833451 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-sys\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833479 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-var-lib-kubelet\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833500 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-host\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833522 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0107a065-c0b7-496d-ad29-8d02b454374c-tmp-dir\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833557 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-sysctl-d\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833554 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833598 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-system-cni-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833633 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833664 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-cni-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833689 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-hostroot\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833714 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/246d90ee-d830-4bf0-b942-ffbfa32a98f9-dbus\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833739 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ceb77fd3-c224-4c26-aad4-923719f14703-serviceca\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833779 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-cni-binary-copy\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.833824 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.833937 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:47.833982 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:48.333967095 +0000 UTC m=+3.066481770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834001 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834076 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-lib-modules\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834160 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0107a065-c0b7-496d-ad29-8d02b454374c-hosts-file\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834210 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-netns\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834245 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-var-lib-cni-bin\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834467 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-system-cni-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834601 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f903377-3e78-436b-8d95-4be29aed46eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834656 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-multus-cni-dir\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834695 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-hostroot\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.834974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.834786 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/246d90ee-d830-4bf0-b942-ffbfa32a98f9-dbus\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.835564 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835187 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ceb77fd3-c224-4c26-aad4-923719f14703-serviceca\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.835635 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835608 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-cni-binary-copy\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.835737 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835688 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-run\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.836027 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835886 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/246d90ee-d830-4bf0-b942-ffbfa32a98f9-kubelet-config\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:47.836027 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835924 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-host-run-k8s-cni-cncf-io\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.836027 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835920 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-var-lib-kubelet\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.836027 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.835999 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-host\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.836310 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.836286 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0107a065-c0b7-496d-ad29-8d02b454374c-tmp-dir\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.836965 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.836919 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f903377-3e78-436b-8d95-4be29aed46eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.839231 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.839203 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 17:26:47.839836 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.839799 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-etc-tuned\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.843567 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.841745 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-tmp\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.849018 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.848996 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp275\" (UniqueName: \"kubernetes.io/projected/a7741c1b-4b2c-4e6a-9d8d-011bc94290be-kube-api-access-zp275\") pod \"tuned-rxxz2\" (UID: \"a7741c1b-4b2c-4e6a-9d8d-011bc94290be\") " pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.850876 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.850728 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6vsb\" (UniqueName: \"kubernetes.io/projected/58726591-2751-4c7d-88c4-b916a4cdc736-kube-api-access-s6vsb\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:47.852888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.852222 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75rpk\" (UniqueName: \"kubernetes.io/projected/7f903377-3e78-436b-8d95-4be29aed46eb-kube-api-access-75rpk\") pod \"multus-additional-cni-plugins-x7pxk\" (UID: \"7f903377-3e78-436b-8d95-4be29aed46eb\") " pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.853823 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.853765 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rm2g\" (UniqueName: \"kubernetes.io/projected/875a08a6-c65f-42a3-9e26-bbdc7c32dbfd-kube-api-access-7rm2g\") pod \"multus-8ktvv\" (UID: \"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd\") " pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.856315 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.856289 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b4zc\" (UniqueName: \"kubernetes.io/projected/ceb77fd3-c224-4c26-aad4-923719f14703-kube-api-access-8b4zc\") pod \"node-ca-8hj5m\" (UID: \"ceb77fd3-c224-4c26-aad4-923719f14703\") " pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.858343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.858303 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9x5n\" (UniqueName: \"kubernetes.io/projected/0107a065-c0b7-496d-ad29-8d02b454374c-kube-api-access-r9x5n\") pod \"node-resolver-mptvx\" (UID: \"0107a065-c0b7-496d-ad29-8d02b454374c\") " pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.925253 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.925222 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:26:47.932237 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.932217 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" Jul 30 17:26:47.933517 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.933468 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd87863_1f53_4966_ba91_7f88ea173688.slice/crio-741e809fafa23e61e6dafa4367b8bb20f9e3d05575910a239885480ee78c738d WatchSource:0}: Error finding container 741e809fafa23e61e6dafa4367b8bb20f9e3d05575910a239885480ee78c738d: Status 404 returned error can't find the container with id 741e809fafa23e61e6dafa4367b8bb20f9e3d05575910a239885480ee78c738d Jul 30 17:26:47.940282 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.940264 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:26:47.947629 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.947561 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4xv95" Jul 30 17:26:47.951002 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.950980 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d14018_07a9_454d_9efd_87d7e2c424fd.slice/crio-6024f9914bbbc3e8674d9643b892aad0c390c1e37d7d4f5006c158ce5a6322a0 WatchSource:0}: Error finding container 6024f9914bbbc3e8674d9643b892aad0c390c1e37d7d4f5006c158ce5a6322a0: Status 404 returned error can't find the container with id 6024f9914bbbc3e8674d9643b892aad0c390c1e37d7d4f5006c158ce5a6322a0 Jul 30 17:26:47.951754 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.951735 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9a6c7b_dbdd_4a56_871d_27aaede78c97.slice/crio-d0783fcb5a144fa8c81dfbec051edc8408fda0ea8ad86d322c1bbebf9befce34 WatchSource:0}: Error finding container d0783fcb5a144fa8c81dfbec051edc8408fda0ea8ad86d322c1bbebf9befce34: Status 404 returned error can't find the container with id d0783fcb5a144fa8c81dfbec051edc8408fda0ea8ad86d322c1bbebf9befce34 Jul 30 17:26:47.954443 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.954424 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" Jul 30 17:26:47.956935 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.956663 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5844c4_84f8_41d2_b3d8_edfd1184fb22.slice/crio-ae12dbe7f25e02755ff3f9afbb3c08df8e714c212fef430bdc94e74e5766867a WatchSource:0}: Error finding container ae12dbe7f25e02755ff3f9afbb3c08df8e714c212fef430bdc94e74e5766867a: Status 404 returned error can't find the container with id ae12dbe7f25e02755ff3f9afbb3c08df8e714c212fef430bdc94e74e5766867a Jul 30 17:26:47.962929 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.962908 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8ktvv" Jul 30 17:26:47.963523 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.963503 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f903377_3e78_436b_8d95_4be29aed46eb.slice/crio-36daa41619bcb34f018e718d63e168e2f957db211b731d11f5026d3f28a6d661 WatchSource:0}: Error finding container 36daa41619bcb34f018e718d63e168e2f957db211b731d11f5026d3f28a6d661: Status 404 returned error can't find the container with id 36daa41619bcb34f018e718d63e168e2f957db211b731d11f5026d3f28a6d661 Jul 30 17:26:47.971904 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.971886 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" Jul 30 17:26:47.974056 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.973950 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875a08a6_c65f_42a3_9e26_bbdc7c32dbfd.slice/crio-ca0648fcefd5d2f91a8ac5a5b2408417b541dc95f5272e697dbe3af428de50b6 WatchSource:0}: Error finding container ca0648fcefd5d2f91a8ac5a5b2408417b541dc95f5272e697dbe3af428de50b6: Status 404 returned error can't find the container with id ca0648fcefd5d2f91a8ac5a5b2408417b541dc95f5272e697dbe3af428de50b6 Jul 30 17:26:47.978716 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.978618 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8hj5m" Jul 30 17:26:47.980014 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.979995 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7741c1b_4b2c_4e6a_9d8d_011bc94290be.slice/crio-792023ccc5482103b24f34fa28df51e22ad3a31a7ad475fb857a0ffe431a004b WatchSource:0}: Error finding container 792023ccc5482103b24f34fa28df51e22ad3a31a7ad475fb857a0ffe431a004b: Status 404 returned error can't find the container with id 792023ccc5482103b24f34fa28df51e22ad3a31a7ad475fb857a0ffe431a004b Jul 30 17:26:47.984921 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:47.984904 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mptvx" Jul 30 17:26:47.994094 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:26:47.994074 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0107a065_c0b7_496d_ad29_8d02b454374c.slice/crio-bf6fb0c3e78832d0d53332085699cc8962dc8d822efcf6b905c250ec961ac635 WatchSource:0}: Error finding container bf6fb0c3e78832d0d53332085699cc8962dc8d822efcf6b905c250ec961ac635: Status 404 returned error can't find the container with id bf6fb0c3e78832d0d53332085699cc8962dc8d822efcf6b905c250ec961ac635 Jul 30 17:26:48.337918 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.337882 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:48.337918 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.337917 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.337964 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338048 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338080 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338109 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338124 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338132 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:49.338110151 +0000 UTC m=+4.070624826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338134 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:48.338166 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338154 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:49.338142666 +0000 UTC m=+4.070657334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:48.338454 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.338185 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:49.338160295 +0000 UTC m=+4.070674959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:48.492409 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.492341 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 17:26:48.662843 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.662761 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 17:21:47 +0000 UTC" deadline="2028-02-28 01:32:33.988241353 +0000 UTC" Jul 30 17:26:48.662843 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.662838 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13856h5m45.325407857s" Jul 30 17:26:48.802645 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.802617 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:48.802829 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:48.802753 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:48.811829 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.811787 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerStarted","Data":"36daa41619bcb34f018e718d63e168e2f957db211b731d11f5026d3f28a6d661"} Jul 30 17:26:48.813062 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.813036 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" event={"ID":"d8d14018-07a9-454d-9efd-87d7e2c424fd","Type":"ContainerStarted","Data":"6024f9914bbbc3e8674d9643b892aad0c390c1e37d7d4f5006c158ce5a6322a0"} Jul 30 17:26:48.814298 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.814273 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wg4tv" event={"ID":"bfd87863-1f53-4966-ba91-7f88ea173688","Type":"ContainerStarted","Data":"741e809fafa23e61e6dafa4367b8bb20f9e3d05575910a239885480ee78c738d"} Jul 30 17:26:48.815559 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.815527 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8hj5m" event={"ID":"ceb77fd3-c224-4c26-aad4-923719f14703","Type":"ContainerStarted","Data":"6f1def0f987458d66448de5cd905ecbb72f825ff3e5e2ee5f6b283153712136d"} Jul 30 17:26:48.816757 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.816728 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4xv95" event={"ID":"0d5844c4-84f8-41d2-b3d8-edfd1184fb22","Type":"ContainerStarted","Data":"ae12dbe7f25e02755ff3f9afbb3c08df8e714c212fef430bdc94e74e5766867a"} Jul 30 17:26:48.818473 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.818447 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"d0783fcb5a144fa8c81dfbec051edc8408fda0ea8ad86d322c1bbebf9befce34"} Jul 30 17:26:48.821955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.821652 2550 generic.go:356] "Generic (PLEG): container finished" podID="c20407b4053ae7e9ce65385b9e6d586a" containerID="52c9e20d1fb042cf3c56b470dfece6330e7455ea72b967e34f005320d34dde5b" exitCode=0 Jul 30 17:26:48.821955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.821752 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" event={"ID":"c20407b4053ae7e9ce65385b9e6d586a","Type":"ContainerDied","Data":"52c9e20d1fb042cf3c56b470dfece6330e7455ea72b967e34f005320d34dde5b"} Jul 30 17:26:48.823348 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.823195 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mptvx" event={"ID":"0107a065-c0b7-496d-ad29-8d02b454374c","Type":"ContainerStarted","Data":"bf6fb0c3e78832d0d53332085699cc8962dc8d822efcf6b905c250ec961ac635"} Jul 30 17:26:48.825334 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.825310 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" event={"ID":"a7741c1b-4b2c-4e6a-9d8d-011bc94290be","Type":"ContainerStarted","Data":"792023ccc5482103b24f34fa28df51e22ad3a31a7ad475fb857a0ffe431a004b"} Jul 30 17:26:48.828529 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:48.828435 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8ktvv" event={"ID":"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd","Type":"ContainerStarted","Data":"ca0648fcefd5d2f91a8ac5a5b2408417b541dc95f5272e697dbe3af428de50b6"} Jul 30 17:26:49.348291 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.348043 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:49.348291 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.348091 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:49.348291 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.348154 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:49.348291 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348291 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348310 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348324 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348381 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:51.348363061 +0000 UTC m=+6.080877746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348453 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348505 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:51.348478203 +0000 UTC m=+6.080992867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348563 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:49.348733 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.348597 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:51.348586648 +0000 UTC m=+6.081101337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:49.663880 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.663729 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 17:21:47 +0000 UTC" deadline="2028-02-22 22:25:20.07008246 +0000 UTC" Jul 30 17:26:49.663880 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.663789 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13732h58m30.406296846s" Jul 30 17:26:49.804860 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.804217 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:49.804860 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.804336 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:26:49.804860 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.804425 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:49.804860 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:49.804498 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:49.849010 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.848357 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" event={"ID":"c20407b4053ae7e9ce65385b9e6d586a","Type":"ContainerStarted","Data":"d7c382346b07a93659cf54e9f25bfb31d3913bea6cbbd40ec3964226cc541211"} Jul 30 17:26:49.856261 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.855919 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" event={"ID":"2288bbe26fd7af2100c8809ffb7ffbce","Type":"ContainerStarted","Data":"c3797ff39cb11cc5d653ec32a796a4cd610968c5e415955d3550e0d0896457a7"} Jul 30 17:26:49.865609 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:49.865478 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-246.ec2.internal" podStartSLOduration=3.865463352 podStartE2EDuration="3.865463352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:26:49.863568645 +0000 UTC m=+4.596083336" watchObservedRunningTime="2026-07-30 17:26:49.865463352 +0000 UTC m=+4.597978037" Jul 30 17:26:50.802229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:50.802197 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:50.802731 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:50.802337 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:51.367664 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:51.367721 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:51.367749 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.367910 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.367971 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:55.367953517 +0000 UTC m=+10.100468182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.368390 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.368413 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.368427 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.368477 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:55.36846149 +0000 UTC m=+10.100976165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.368536 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:51.368632 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.368569 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:55.368558726 +0000 UTC m=+10.101073389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:51.804973 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:51.804895 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:51.805411 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.805023 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:26:51.805411 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:51.805083 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:51.805411 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:51.805152 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:52.803176 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:52.802702 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:52.803176 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:52.802858 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:53.813949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:53.813907 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:53.814447 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:53.814088 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:53.814447 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:53.814204 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:53.814447 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:53.814283 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:26:54.802691 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:54.802131 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:54.802691 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:54.802278 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:55.402556 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:55.402517 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:55.402556 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:55.402563 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:55.402614 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.402757 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.402774 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.402787 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.402865 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:03.402844941 +0000 UTC m=+18.135359622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.402940 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.402974 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:03.402963924 +0000 UTC m=+18.135478585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.403022 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:55.403079 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.403053 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:03.403041713 +0000 UTC m=+18.135556374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:26:55.803592 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:55.803525 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:55.803592 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:55.803586 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:55.803767 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.803677 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:55.803767 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:55.803725 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:26:56.802157 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:56.802055 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:56.802591 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:56.802177 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:57.802850 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:57.802794 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:57.803265 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:57.802946 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:26:57.803265 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:57.802955 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:57.803377 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:57.803271 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:58.802472 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:58.802444 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:26:58.802618 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:58.802545 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:26:59.805165 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:59.805134 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:26:59.805165 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:26:59.805156 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:26:59.805551 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:59.805248 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:26:59.805551 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:26:59.805385 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:00.802328 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:00.802297 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:00.802484 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:00.802417 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:01.802866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:01.802828 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:01.803267 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:01.802884 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:01.803267 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:01.802976 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:01.803267 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:01.803108 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:02.802687 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:02.802654 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:02.802904 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:02.802802 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:03.465834 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:03.465738 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:03.465834 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:03.465791 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:03.465834 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:03.465830 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.465924 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.465949 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.465955 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.465959 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.465964 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.466027 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:19.466005849 +0000 UTC m=+34.198520530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.466044 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:19.466035625 +0000 UTC m=+34.198550290 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:27:03.466154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.466063 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:19.466054386 +0000 UTC m=+34.198569050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:27:03.802276 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:03.802199 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:03.802434 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.802313 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:03.802434 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:03.802377 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:03.802562 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:03.802480 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:04.801864 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:04.801834 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:04.802301 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:04.801957 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:05.802699 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:05.802668 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:05.803179 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:05.802763 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:05.803179 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:05.802765 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:05.803179 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:05.802895 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:06.802419 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:06.802382 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:06.802615 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:06.802526 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:07.802180 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.801904 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:07.802716 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.801911 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:07.802716 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:07.802262 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:07.802716 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:07.802350 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:07.894589 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.894563 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8ktvv" event={"ID":"875a08a6-c65f-42a3-9e26-bbdc7c32dbfd","Type":"ContainerStarted","Data":"229b1ee146fa782e11c3a6b0d516ec379325e782a91eec8b9ba1464eb8660925"} Jul 30 17:27:07.895981 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.895956 2550 generic.go:356] "Generic (PLEG): container finished" podID="7f903377-3e78-436b-8d95-4be29aed46eb" containerID="cda090a0c945bdbb79aef7f94512997f697b0d2110ad3a12c0adc3883e5f99a7" exitCode=0 Jul 30 17:27:07.896063 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.896035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerDied","Data":"cda090a0c945bdbb79aef7f94512997f697b0d2110ad3a12c0adc3883e5f99a7"} Jul 30 17:27:07.897374 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.897337 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" event={"ID":"d8d14018-07a9-454d-9efd-87d7e2c424fd","Type":"ContainerStarted","Data":"ea284ae64b13de096c30a129ee359ed1511ec04e39d9dd106b455f52cec1099f"} Jul 30 17:27:07.898647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.898613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wg4tv" event={"ID":"bfd87863-1f53-4966-ba91-7f88ea173688","Type":"ContainerStarted","Data":"dc752a008c6d6d4bab88776f0bef3d0680b287c37db43f6d39be676ce95b230a"} Jul 30 17:27:07.899899 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.899874 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8hj5m" event={"ID":"ceb77fd3-c224-4c26-aad4-923719f14703","Type":"ContainerStarted","Data":"1fdb11424f20a862856fb19aaf40556b65ff2beb8991fdad713bace86f69413b"} Jul 30 17:27:07.902040 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.902018 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"992a77eeba494755019c4794c75811c31bdcf10d62cd532726ec9cad0d0b94fb"} Jul 30 17:27:07.902123 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.902093 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"432296d75130857d0cbb34effc3cf821fcc696c0d0d02f9739ad68c4f4cc2d39"} Jul 30 17:27:07.902183 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.902139 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"239e7085a0c5fe74c4af25b07ac2d13d97097197223aab8efe0b9b384de5a0c4"} Jul 30 17:27:07.903271 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.903252 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mptvx" event={"ID":"0107a065-c0b7-496d-ad29-8d02b454374c","Type":"ContainerStarted","Data":"f9682b25d39996fd00d94e370d0b3a8a72f58e4bd1c4e4017138cb042dd6e25c"} Jul 30 17:27:07.904428 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.904410 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" event={"ID":"a7741c1b-4b2c-4e6a-9d8d-011bc94290be","Type":"ContainerStarted","Data":"cea46eefc0c5405cf44322ff3f42e47453baf08fcb7893485354824b93e9cc72"} Jul 30 17:27:07.923190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.923150 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8ktvv" podStartSLOduration=2.49961498 podStartE2EDuration="21.923139179s" totalImagesPullingTime="19.423524199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:07.922695346 +0000 UTC m=+22.655210030" watchObservedRunningTime="2026-07-30 17:27:07.923139179 +0000 UTC m=+22.655653863" Jul 30 17:27:07.923396 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.923345 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-246.ec2.internal" podStartSLOduration=21.923340366 podStartE2EDuration="21.923340366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:26:49.877006476 +0000 UTC m=+4.609521160" watchObservedRunningTime="2026-07-30 17:27:07.923340366 +0000 UTC m=+22.655855083" Jul 30 17:27:07.971583 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.971544 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rxxz2" podStartSLOduration=2.597482911 podStartE2EDuration="21.971529944s" totalImagesPullingTime="19.374047033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:07.945294994 +0000 UTC m=+22.677809677" watchObservedRunningTime="2026-07-30 17:27:07.971529944 +0000 UTC m=+22.704044627" Jul 30 17:27:07.971669 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:07.971637 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mptvx" podStartSLOduration=2.612259749 podStartE2EDuration="21.971631871s" totalImagesPullingTime="19.359372122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:07.971511667 +0000 UTC m=+22.704026352" watchObservedRunningTime="2026-07-30 17:27:07.971631871 +0000 UTC m=+22.704146555" Jul 30 17:27:08.003149 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.003113 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wg4tv" podStartSLOduration=7.747150688 podStartE2EDuration="23.003098577s" totalImagesPullingTime="15.255947889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:08.002253708 +0000 UTC m=+22.734768391" watchObservedRunningTime="2026-07-30 17:27:08.003098577 +0000 UTC m=+22.735613259" Jul 30 17:27:08.120591 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.120536 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8hj5m" podStartSLOduration=2.752760854 podStartE2EDuration="22.120515159s" totalImagesPullingTime="19.367754305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:08.034079307 +0000 UTC m=+22.766593991" watchObservedRunningTime="2026-07-30 17:27:08.120515159 +0000 UTC m=+22.853029844" Jul 30 17:27:08.802044 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.802018 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:08.802181 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:08.802129 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:08.858192 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.858168 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 17:27:08.910401 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.910162 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"717e92c26b5201c8a000a0e794b4ba5468bb2569c6ca3ebb7ce9dbbc31eca73b"} Jul 30 17:27:08.910401 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.910264 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"aa4adbce826b1f4bccea90a4a580032ad5b67b00aa386c8599ce0f8194f944f7"} Jul 30 17:27:08.910401 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.910314 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"b7770ca5db09bba14de0ffd23a1687da6995d7cd2b828aa62d87856a99873ab1"} Jul 30 17:27:08.912012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.911985 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" event={"ID":"d8d14018-07a9-454d-9efd-87d7e2c424fd","Type":"ContainerStarted","Data":"e11890d1964f8634084536c230e773a2dcb7132db017adf1b5306112a2e5732c"} Jul 30 17:27:08.916639 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.913394 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4xv95" event={"ID":"0d5844c4-84f8-41d2-b3d8-edfd1184fb22","Type":"ContainerStarted","Data":"38ebefaf073022673cf89b996cb4f31c5a0d2291df08001f57922fdb58898bd6"} Jul 30 17:27:08.935109 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:08.935069 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4xv95" podStartSLOduration=4.602630065 podStartE2EDuration="23.935056338s" totalImagesPullingTime="19.332426273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:08.934710315 +0000 UTC m=+23.667225000" watchObservedRunningTime="2026-07-30 17:27:08.935056338 +0000 UTC m=+23.667571023" Jul 30 17:27:09.687067 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:09.686956 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T17:27:08.858188649Z","UUID":"55c5f6d7-1a62-4180-96dc-f1a6004c6948","Handler":null,"Name":"","Endpoint":""} Jul 30 17:27:09.689603 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:09.689579 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 17:27:09.689733 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:09.689625 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 17:27:09.802799 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:09.802772 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:09.802992 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:09.802904 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:09.803398 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:09.803376 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:09.803504 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:09.803476 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:10.802161 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:10.801969 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:10.802538 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:10.802248 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:10.921749 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:10.921709 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"2fc283a41ff3426f5ab82326a9238578390efda70cf65f5fb0a8b502181fb9e3"} Jul 30 17:27:10.924102 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:10.924064 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" event={"ID":"d8d14018-07a9-454d-9efd-87d7e2c424fd","Type":"ContainerStarted","Data":"fabaca053d73eff69ad6e62e3f18dfe2e8496e02bfcdaaf84169e50e475e39d5"} Jul 30 17:27:11.802550 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:11.802516 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:11.803131 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:11.802654 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:11.803131 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:11.802688 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:11.803131 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:11.802804 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:12.281478 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.281438 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:27:12.282152 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.282117 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:27:12.306199 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.306163 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qf6p4" podStartSLOduration=5.609891958 podStartE2EDuration="27.306150283s" totalImagesPullingTime="21.696258325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:10.950334982 +0000 UTC m=+25.682849678" watchObservedRunningTime="2026-07-30 17:27:12.306150283 +0000 UTC m=+27.038664967" Jul 30 17:27:12.802316 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.802075 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:12.802461 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:12.802335 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:12.929624 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.929592 2550 generic.go:356] "Generic (PLEG): container finished" podID="7f903377-3e78-436b-8d95-4be29aed46eb" containerID="2329f87617b492bb9c007ec17e0ee01d0ebc0a2ebdf7acb8788055df285d2eb7" exitCode=0 Jul 30 17:27:12.930363 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.929681 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerDied","Data":"2329f87617b492bb9c007ec17e0ee01d0ebc0a2ebdf7acb8788055df285d2eb7"} Jul 30 17:27:12.933546 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.933520 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" event={"ID":"8d9a6c7b-dbdd-4a56-871d-27aaede78c97","Type":"ContainerStarted","Data":"fec4d1c854d05d04151a040211e4bcfdfb9b0a4205d694656b213faf776ff922"} Jul 30 17:27:12.934005 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.933981 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:27:12.934289 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:12.934273 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wg4tv" Jul 30 17:27:13.805713 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.805679 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:13.805897 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:13.805858 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:13.806058 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.806037 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:13.806160 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:13.806138 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:13.936206 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.936180 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:27:13.936611 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.936273 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:27:13.936611 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.936329 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:27:13.954665 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.954642 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:27:13.955192 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.955166 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:27:13.994673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:13.994632 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" podStartSLOduration=9.441051168 podStartE2EDuration="28.994618505s" totalImagesPullingTime="19.553567337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:13.005698372 +0000 UTC m=+27.738213056" watchObservedRunningTime="2026-07-30 17:27:13.994618505 +0000 UTC m=+28.727133189" Jul 30 17:27:14.802166 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:14.802136 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:14.802296 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:14.802237 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:14.939858 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:14.939806 2550 generic.go:356] "Generic (PLEG): container finished" podID="7f903377-3e78-436b-8d95-4be29aed46eb" containerID="79477a42e8f4cac387c4137294055c560573365e9e4393430b67d3da6e74e027" exitCode=0 Jul 30 17:27:14.940259 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:14.939907 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerDied","Data":"79477a42e8f4cac387c4137294055c560573365e9e4393430b67d3da6e74e027"} Jul 30 17:27:15.116189 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:15.116151 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8lrfj"] Jul 30 17:27:15.116337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:15.116281 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:15.116386 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:15.116362 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:15.120067 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:15.120043 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8nlrg"] Jul 30 17:27:15.120216 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:15.120132 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:15.120268 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:15.120229 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:15.125043 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:15.125018 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9b5ft"] Jul 30 17:27:15.125174 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:15.125110 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:15.125230 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:15.125180 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:16.802233 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:16.802018 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:16.802587 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:16.802060 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:16.802587 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:16.802275 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:16.802587 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:16.802072 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:16.802587 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:16.802330 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:16.802587 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:16.802410 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:16.946857 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:16.946806 2550 generic.go:356] "Generic (PLEG): container finished" podID="7f903377-3e78-436b-8d95-4be29aed46eb" containerID="1c748ec425ca17c53597ae6fe46d5f60637b7c18f8c7eb6365ad287fbef7d22b" exitCode=0 Jul 30 17:27:16.947015 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:16.946856 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerDied","Data":"1c748ec425ca17c53597ae6fe46d5f60637b7c18f8c7eb6365ad287fbef7d22b"} Jul 30 17:27:18.802783 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:18.802747 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:18.802783 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:18.802772 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:18.803363 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:18.802775 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:18.803363 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:18.802905 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:18.803363 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:18.803022 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:18.803363 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:18.803105 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:19.487392 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:19.487356 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:19.487551 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:19.487399 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:19.487551 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487497 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 17:27:19.487551 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487542 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret podName:246d90ee-d830-4bf0-b942-ffbfa32a98f9 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:51.487529105 +0000 UTC m=+66.220043766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret") pod "global-pull-secret-syncer-8lrfj" (UID: "246d90ee-d830-4bf0-b942-ffbfa32a98f9") : object "kube-system"/"original-pull-secret" not registered Jul 30 17:27:19.487657 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487626 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 17:27:19.487657 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487646 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 17:27:19.487657 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487655 2550 projected.go:196] Error preparing data for projected volume kube-api-access-qv2zg for pod openshift-network-diagnostics/network-check-target-9b5ft: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:27:19.487761 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:19.487658 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:19.487761 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487692 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg podName:6fabf24e-bb0e-4c45-b3ae-7b7c96179177 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:51.487680982 +0000 UTC m=+66.220195644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv2zg" (UniqueName: "kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg") pod "network-check-target-9b5ft" (UID: "6fabf24e-bb0e-4c45-b3ae-7b7c96179177") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 17:27:19.487761 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487736 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:27:19.487897 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:19.487779 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:51.487766544 +0000 UTC m=+66.220281221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 17:27:20.802778 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:20.802748 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:20.803352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:20.802787 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:20.803352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:20.802901 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:20.803352 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:20.802898 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:27:20.803352 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:20.802985 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9b5ft" podUID="6fabf24e-bb0e-4c45-b3ae-7b7c96179177" Jul 30 17:27:20.803352 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:20.803097 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lrfj" podUID="246d90ee-d830-4bf0-b942-ffbfa32a98f9" Jul 30 17:27:21.048655 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.048629 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-246.ec2.internal" event="NodeReady" Jul 30 17:27:21.048861 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.048783 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 17:27:21.098005 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.097969 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk"] Jul 30 17:27:21.133507 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.133484 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f58ff6cff-zx8p4"] Jul 30 17:27:21.133662 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.133620 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.136260 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.136238 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 30 17:27:21.136370 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.136258 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mnxwp\"" Jul 30 17:27:21.136965 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.136947 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 30 17:27:21.147447 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.147424 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk"] Jul 30 17:27:21.147541 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.147464 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f58ff6cff-zx8p4"] Jul 30 17:27:21.147541 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.147485 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5g9cd"] Jul 30 17:27:21.147639 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.147546 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.154346 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.154327 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 17:27:21.154531 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.154513 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 17:27:21.154640 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.154574 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 17:27:21.154640 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.154614 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bns8l\"" Jul 30 17:27:21.162101 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.162082 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 17:27:21.169516 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.169499 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tb9js"] Jul 30 17:27:21.169635 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.169618 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.175034 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.174577 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 17:27:21.175034 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.174692 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 17:27:21.175034 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.174698 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sr7s9\"" Jul 30 17:27:21.175034 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.174933 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 17:27:21.187582 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.187559 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tb9js"] Jul 30 17:27:21.187684 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.187605 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5g9cd"] Jul 30 17:27:21.187744 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.187698 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.192538 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.192518 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 17:27:21.192702 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.192678 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tjl7j\"" Jul 30 17:27:21.192871 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.192858 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 17:27:21.300292 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300257 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df4a595-c599-46ba-aa60-d2dc6bd50806-tmp-dir\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.300292 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300294 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ld8v\" (UniqueName: \"kubernetes.io/projected/1743b5d7-6896-4df5-9535-49be6477dc1d-kube-api-access-7ld8v\") pod \"ingress-canary-5g9cd\" (UID: \"1743b5d7-6896-4df5-9535-49be6477dc1d\") " pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.300530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300332 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-installation-pull-secrets\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300371 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0df4a595-c599-46ba-aa60-d2dc6bd50806-config-volume\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.300530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/53ddc810-ac04-4499-a5b1-bc820b719759-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.300530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300455 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-image-registry-private-configuration\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300472 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r627m\" (UniqueName: \"kubernetes.io/projected/0df4a595-c599-46ba-aa60-d2dc6bd50806-kube-api-access-r627m\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.300530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300486 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1743b5d7-6896-4df5-9535-49be6477dc1d-cert\") pod \"ingress-canary-5g9cd\" (UID: \"1743b5d7-6896-4df5-9535-49be6477dc1d\") " pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300544 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300558 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-certificates\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300573 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd405999-57f1-4712-8fb0-6d19aa07f528-ca-trust-extracted\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300586 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-trusted-ca\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300621 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-bound-sa-token\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300660 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpvn5\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-kube-api-access-mpvn5\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300674 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.300842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.300688 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.401672 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401612 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.401672 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401642 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-certificates\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.401672 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401661 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd405999-57f1-4712-8fb0-6d19aa07f528-ca-trust-extracted\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401677 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-trusted-ca\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401714 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-bound-sa-token\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401755 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpvn5\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-kube-api-access-mpvn5\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401779 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401801 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.401824 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.401845 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401848 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df4a595-c599-46ba-aa60-d2dc6bd50806-tmp-dir\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401875 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ld8v\" (UniqueName: \"kubernetes.io/projected/1743b5d7-6896-4df5-9535-49be6477dc1d-kube-api-access-7ld8v\") pod \"ingress-canary-5g9cd\" (UID: \"1743b5d7-6896-4df5-9535-49be6477dc1d\") " pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.401941 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.401911 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:21.901888903 +0000 UTC m=+36.634403583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401964 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-installation-pull-secrets\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.401996 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0df4a595-c599-46ba-aa60-d2dc6bd50806-config-volume\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402041 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/53ddc810-ac04-4499-a5b1-bc820b719759-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402069 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-image-registry-private-configuration\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402088 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd405999-57f1-4712-8fb0-6d19aa07f528-ca-trust-extracted\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.402095 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.402178 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:21.902157488 +0000 UTC m=+36.634672153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402098 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r627m\" (UniqueName: \"kubernetes.io/projected/0df4a595-c599-46ba-aa60-d2dc6bd50806-kube-api-access-r627m\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402215 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1743b5d7-6896-4df5-9535-49be6477dc1d-cert\") pod \"ingress-canary-5g9cd\" (UID: \"1743b5d7-6896-4df5-9535-49be6477dc1d\") " pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402398 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df4a595-c599-46ba-aa60-d2dc6bd50806-tmp-dir\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.402475 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.402474 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:21.402988 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.402515 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:21.902501995 +0000 UTC m=+36.635016656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:21.402988 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402641 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-trusted-ca\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.402988 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402761 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0df4a595-c599-46ba-aa60-d2dc6bd50806-config-volume\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.402988 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402891 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-certificates\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.402988 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.402889 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/53ddc810-ac04-4499-a5b1-bc820b719759-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.406195 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.406168 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-installation-pull-secrets\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.406195 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.406177 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-image-registry-private-configuration\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.406324 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.406211 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1743b5d7-6896-4df5-9535-49be6477dc1d-cert\") pod \"ingress-canary-5g9cd\" (UID: \"1743b5d7-6896-4df5-9535-49be6477dc1d\") " pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.417266 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.417245 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r627m\" (UniqueName: \"kubernetes.io/projected/0df4a595-c599-46ba-aa60-d2dc6bd50806-kube-api-access-r627m\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.418214 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.418193 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-bound-sa-token\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.420564 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.420542 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpvn5\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-kube-api-access-mpvn5\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.420898 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.420879 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ld8v\" (UniqueName: \"kubernetes.io/projected/1743b5d7-6896-4df5-9535-49be6477dc1d-kube-api-access-7ld8v\") pod \"ingress-canary-5g9cd\" (UID: \"1743b5d7-6896-4df5-9535-49be6477dc1d\") " pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.481441 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.481419 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5g9cd" Jul 30 17:27:21.666195 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.666004 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5g9cd"] Jul 30 17:27:21.669610 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:27:21.669584 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1743b5d7_6896_4df5_9535_49be6477dc1d.slice/crio-a1b383be7562d3246b4549ff618ad8a7e71237548a2546521905c41d8a355ac9 WatchSource:0}: Error finding container a1b383be7562d3246b4549ff618ad8a7e71237548a2546521905c41d8a355ac9: Status 404 returned error can't find the container with id a1b383be7562d3246b4549ff618ad8a7e71237548a2546521905c41d8a355ac9 Jul 30 17:27:21.905274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.905241 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.905310 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.905339 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905391 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905413 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905452 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905470 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:22.905446216 +0000 UTC m=+37.637960916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905486 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905503 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:22.905489433 +0000 UTC m=+37.638004098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:21.905734 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:21.905527 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:22.905515542 +0000 UTC m=+37.638030219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:21.958730 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.958667 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5g9cd" event={"ID":"1743b5d7-6896-4df5-9535-49be6477dc1d","Type":"ContainerStarted","Data":"a1b383be7562d3246b4549ff618ad8a7e71237548a2546521905c41d8a355ac9"} Jul 30 17:27:21.961172 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.961153 2550 generic.go:356] "Generic (PLEG): container finished" podID="7f903377-3e78-436b-8d95-4be29aed46eb" containerID="6e4c60801948f64bebc5bfa48e12b7b8f152b38983450f629dd27985f3e5b1d7" exitCode=0 Jul 30 17:27:21.961253 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:21.961185 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerDied","Data":"6e4c60801948f64bebc5bfa48e12b7b8f152b38983450f629dd27985f3e5b1d7"} Jul 30 17:27:22.802221 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.802183 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:22.802475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.802184 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:22.802475 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.802184 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:22.805677 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.805376 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 17:27:22.805677 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.805393 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 17:27:22.805677 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.805380 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 17:27:22.805677 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.805467 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 17:27:22.806775 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.806756 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjwv8\"" Jul 30 17:27:22.806880 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.806827 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qqrb\"" Jul 30 17:27:22.913955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.913925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:22.914358 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.913996 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:22.914358 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.914020 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:22.914358 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914113 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:22.914358 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914165 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:24.914152057 +0000 UTC m=+39.646666718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:22.914594 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914452 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:22.914594 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914462 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:22.914594 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914499 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:22.914594 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914523 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:24.914502639 +0000 UTC m=+39.647017323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:22.914594 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:22.914547 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:24.914536459 +0000 UTC m=+39.647051125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:22.968147 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.968120 2550 generic.go:356] "Generic (PLEG): container finished" podID="7f903377-3e78-436b-8d95-4be29aed46eb" containerID="9b34a412f7f96ff494a30b429550fa91d401de65015371d9bd00ad601d03e88b" exitCode=0 Jul 30 17:27:22.968316 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:22.968192 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerDied","Data":"9b34a412f7f96ff494a30b429550fa91d401de65015371d9bd00ad601d03e88b"} Jul 30 17:27:23.974308 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:23.974118 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" event={"ID":"7f903377-3e78-436b-8d95-4be29aed46eb","Type":"ContainerStarted","Data":"68eaec00f111e5afbf8155b18de2a3b7350f248729ddeed9bf8b7e33a81dbea5"} Jul 30 17:27:24.008979 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:24.008930 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x7pxk" podStartSLOduration=7.539932679 podStartE2EDuration="38.008911071s" totalImagesPullingTime="30.468978392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:24.008167173 +0000 UTC m=+38.740681860" watchObservedRunningTime="2026-07-30 17:27:24.008911071 +0000 UTC m=+38.741425755" Jul 30 17:27:24.931040 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:24.930986 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:24.931196 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:24.931049 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:24.931196 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:24.931127 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:24.931196 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931149 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:24.931294 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931216 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:28.931195732 +0000 UTC m=+43.663710411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:24.931294 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931217 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:24.931294 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931232 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:24.931294 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931213 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:24.931420 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931262 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:28.931252964 +0000 UTC m=+43.663767629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:24.931420 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:24.931321 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:28.931306646 +0000 UTC m=+43.663821307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:24.978216 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:24.978149 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5g9cd" event={"ID":"1743b5d7-6896-4df5-9535-49be6477dc1d","Type":"ContainerStarted","Data":"6054d5f760435c86d418769251530e365b9b5d8f8181068229c19841d2819203"} Jul 30 17:27:25.001947 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:25.001912 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5g9cd" podStartSLOduration=1.013091065 podStartE2EDuration="4.00189743s" totalImagesPullingTime="2.988806365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:25.001498598 +0000 UTC m=+39.734013281" watchObservedRunningTime="2026-07-30 17:27:25.00189743 +0000 UTC m=+39.734412114" Jul 30 17:27:28.962789 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:28.962753 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:28.962789 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:28.962790 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:28.962857 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.962927 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.962945 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.962955 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.962966 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.963008 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:36.962988537 +0000 UTC m=+51.695503213 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.963028 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:36.963018669 +0000 UTC m=+51.695533333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:28.963287 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:28.963045 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:36.963036941 +0000 UTC m=+51.695551607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:37.025339 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:37.025303 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:37.025370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:37.025396 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025463 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025478 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025518 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025531 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025536 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:53.025517157 +0000 UTC m=+67.758031834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025607 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:53.025589863 +0000 UTC m=+67.758104528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:37.025747 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:37.025620 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:53.025613189 +0000 UTC m=+67.758127851 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:45.958785 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:45.958759 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m8qn9" Jul 30 17:27:51.528849 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.528793 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:51.528849 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.528850 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:27:51.529381 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.528899 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:51.532164 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.532144 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 17:27:51.532230 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.532148 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 17:27:51.532352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.532340 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 17:27:51.539627 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:51.539605 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 17:27:51.539715 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:51.539672 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:28:55.539651759 +0000 UTC m=+130.272166435 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : secret "metrics-daemon-secret" not found Jul 30 17:27:51.542279 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.542264 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 17:27:51.543381 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.543360 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/246d90ee-d830-4bf0-b942-ffbfa32a98f9-original-pull-secret\") pod \"global-pull-secret-syncer-8lrfj\" (UID: \"246d90ee-d830-4bf0-b942-ffbfa32a98f9\") " pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:51.552780 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.552759 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv2zg\" (UniqueName: \"kubernetes.io/projected/6fabf24e-bb0e-4c45-b3ae-7b7c96179177-kube-api-access-qv2zg\") pod \"network-check-target-9b5ft\" (UID: \"6fabf24e-bb0e-4c45-b3ae-7b7c96179177\") " pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:51.616079 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.616055 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lrfj" Jul 30 17:27:51.631472 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.631453 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qqrb\"" Jul 30 17:27:51.638400 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.638383 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:51.771305 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.771279 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8lrfj"] Jul 30 17:27:51.774578 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:27:51.774553 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246d90ee_d830_4bf0_b942_ffbfa32a98f9.slice/crio-23830a1c9c2cce4585a0c12baf35f8051a29c6965465b284ce35f07b1efa5676 WatchSource:0}: Error finding container 23830a1c9c2cce4585a0c12baf35f8051a29c6965465b284ce35f07b1efa5676: Status 404 returned error can't find the container with id 23830a1c9c2cce4585a0c12baf35f8051a29c6965465b284ce35f07b1efa5676 Jul 30 17:27:51.791149 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:51.791099 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9b5ft"] Jul 30 17:27:51.794477 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:27:51.794451 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fabf24e_bb0e_4c45_b3ae_7b7c96179177.slice/crio-d766e7ce1c36892c965a9196c70deb15c1c45c5e7d9dcbe5eb3e09eadb09136e WatchSource:0}: Error finding container d766e7ce1c36892c965a9196c70deb15c1c45c5e7d9dcbe5eb3e09eadb09136e: Status 404 returned error can't find the container with id d766e7ce1c36892c965a9196c70deb15c1c45c5e7d9dcbe5eb3e09eadb09136e Jul 30 17:27:52.040263 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:52.040225 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8lrfj" event={"ID":"246d90ee-d830-4bf0-b942-ffbfa32a98f9","Type":"ContainerStarted","Data":"23830a1c9c2cce4585a0c12baf35f8051a29c6965465b284ce35f07b1efa5676"} Jul 30 17:27:52.041270 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:52.041205 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9b5ft" event={"ID":"6fabf24e-bb0e-4c45-b3ae-7b7c96179177","Type":"ContainerStarted","Data":"d766e7ce1c36892c965a9196c70deb15c1c45c5e7d9dcbe5eb3e09eadb09136e"} Jul 30 17:27:53.038947 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:53.038917 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:53.038954 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:53.039003 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039120 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039123 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039141 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039144 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039184 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:28:25.039164925 +0000 UTC m=+99.771679600 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039198 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:28:25.039192731 +0000 UTC m=+99.771707392 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:27:53.039371 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:27:53.039208 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:28:25.039203117 +0000 UTC m=+99.771717778 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:27:57.056037 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:57.056002 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8lrfj" event={"ID":"246d90ee-d830-4bf0-b942-ffbfa32a98f9","Type":"ContainerStarted","Data":"e41ea59e7e202039e643ccdfaddea41a5e13db6770abc37f346c5eaeb37caff2"} Jul 30 17:27:57.057476 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:57.057452 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9b5ft" event={"ID":"6fabf24e-bb0e-4c45-b3ae-7b7c96179177","Type":"ContainerStarted","Data":"6b612eea9d253b2e1586b1028a68759701c8fdd12485553daf9ea21c4c9db3c5"} Jul 30 17:27:57.057601 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:57.057582 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:27:57.077172 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:57.077128 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8lrfj" podStartSLOduration=66.650769487 podStartE2EDuration="1m11.077115632s" totalImagesPullingTime="4.426346145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:57.076899552 +0000 UTC m=+71.809414233" watchObservedRunningTime="2026-07-30 17:27:57.077115632 +0000 UTC m=+71.809630316" Jul 30 17:27:57.097976 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:27:57.097939 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9b5ft" podStartSLOduration=67.694836934 podStartE2EDuration="1m12.097923095s" totalImagesPullingTime="4.403086161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:57.097918209 +0000 UTC m=+71.830432893" watchObservedRunningTime="2026-07-30 17:27:57.097923095 +0000 UTC m=+71.830437963" Jul 30 17:28:04.446414 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.446382 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65"] Jul 30 17:28:04.448286 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.448268 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.452929 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.452908 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 30 17:28:04.453075 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.452932 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 30 17:28:04.453075 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.452993 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 30 17:28:04.453162 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.453101 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 30 17:28:04.468369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.468350 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65"] Jul 30 17:28:04.521859 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.521823 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/128a0597-24f2-4a75-853f-9af2719166b0-tmp\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.522003 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.521918 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzzsg\" (UniqueName: \"kubernetes.io/projected/128a0597-24f2-4a75-853f-9af2719166b0-kube-api-access-zzzsg\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.522003 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.521978 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/128a0597-24f2-4a75-853f-9af2719166b0-klusterlet-config\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.622433 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.622382 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzzsg\" (UniqueName: \"kubernetes.io/projected/128a0597-24f2-4a75-853f-9af2719166b0-kube-api-access-zzzsg\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.622566 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.622484 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/128a0597-24f2-4a75-853f-9af2719166b0-klusterlet-config\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.622566 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.622534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/128a0597-24f2-4a75-853f-9af2719166b0-tmp\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.622968 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.622947 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/128a0597-24f2-4a75-853f-9af2719166b0-tmp\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.625080 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.625058 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/128a0597-24f2-4a75-853f-9af2719166b0-klusterlet-config\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.631728 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.631710 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzzsg\" (UniqueName: \"kubernetes.io/projected/128a0597-24f2-4a75-853f-9af2719166b0-kube-api-access-zzzsg\") pod \"klusterlet-addon-workmgr-6899f45f4b-b7g65\" (UID: \"128a0597-24f2-4a75-853f-9af2719166b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.757117 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.757036 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:04.883912 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:04.883885 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65"] Jul 30 17:28:04.886932 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:28:04.886900 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128a0597_24f2_4a75_853f_9af2719166b0.slice/crio-8737afbe881f72a2ef36af799f3f490d47a9bef9a7e9796b9fc2b9011546e533 WatchSource:0}: Error finding container 8737afbe881f72a2ef36af799f3f490d47a9bef9a7e9796b9fc2b9011546e533: Status 404 returned error can't find the container with id 8737afbe881f72a2ef36af799f3f490d47a9bef9a7e9796b9fc2b9011546e533 Jul 30 17:28:05.078106 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:05.078033 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" event={"ID":"128a0597-24f2-4a75-853f-9af2719166b0","Type":"ContainerStarted","Data":"8737afbe881f72a2ef36af799f3f490d47a9bef9a7e9796b9fc2b9011546e533"} Jul 30 17:28:09.091858 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:09.091824 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" event={"ID":"128a0597-24f2-4a75-853f-9af2719166b0","Type":"ContainerStarted","Data":"c07ffc1dae0b62ba1727af6db63fe680140252d7ba6ccaba98f4d24ca0ccfc6a"} Jul 30 17:28:09.092327 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:09.092032 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:09.093543 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:09.093525 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:28:09.127072 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:09.127037 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" podStartSLOduration=1.634694935 podStartE2EDuration="5.127025749s" totalImagesPullingTime="3.492330814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:28:09.110265162 +0000 UTC m=+83.842779847" watchObservedRunningTime="2026-07-30 17:28:09.127025749 +0000 UTC m=+83.859540460" Jul 30 17:28:25.073665 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:25.073634 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:25.073680 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:25.073700 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073782 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073788 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073796 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073839 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073853 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:29:29.073839188 +0000 UTC m=+163.806353850 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073865 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:29:29.073859748 +0000 UTC m=+163.806374410 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:28:25.074036 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:25.073893 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:29:29.073877479 +0000 UTC m=+163.806392166 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:28:28.063054 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:28.063019 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9b5ft" Jul 30 17:28:55.588026 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:28:55.587991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:28:55.588482 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:55.588117 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 17:28:55.588482 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:28:55.588168 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:30:57.588153833 +0000 UTC m=+252.320668495 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : secret "metrics-daemon-secret" not found Jul 30 17:29:24.145000 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:24.144952 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" podUID="53ddc810-ac04-4499-a5b1-bc820b719759" Jul 30 17:29:24.157170 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:24.157141 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" Jul 30 17:29:24.197510 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:24.197487 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tb9js" podUID="0df4a595-c599-46ba-aa60-d2dc6bd50806" Jul 30 17:29:24.261851 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:24.261832 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:29:25.821438 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:25.821402 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:29:29.117676 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:29.117623 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:29.117703 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:29.117734 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117774 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117792 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f58ff6cff-zx8p4: secret "image-registry-tls" not found Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117843 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117885 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117890 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls podName:fd405999-57f1-4712-8fb0-6d19aa07f528 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:31.117869628 +0000 UTC m=+285.850384337 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls") pod "image-registry-5f58ff6cff-zx8p4" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528") : secret "image-registry-tls" not found Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117965 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:31.117946884 +0000 UTC m=+285.850461549 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:29:29.118229 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:29:29.117984 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls podName:0df4a595-c599-46ba-aa60-d2dc6bd50806 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:31.117976916 +0000 UTC m=+285.850491579 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls") pod "dns-default-tb9js" (UID: "0df4a595-c599-46ba-aa60-d2dc6bd50806") : secret "dns-default-metrics-tls" not found Jul 30 17:29:35.803040 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:35.803002 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:29:39.802673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:39.802581 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:29:39.802673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:29:39.802635 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9js" Jul 30 17:30:09.093070 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:09.093013 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" podUID="128a0597-24f2-4a75-853f-9af2719166b0" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused" Jul 30 17:30:09.367697 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:09.367619 2550 generic.go:356] "Generic (PLEG): container finished" podID="128a0597-24f2-4a75-853f-9af2719166b0" containerID="c07ffc1dae0b62ba1727af6db63fe680140252d7ba6ccaba98f4d24ca0ccfc6a" exitCode=1 Jul 30 17:30:09.367857 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:09.367692 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" event={"ID":"128a0597-24f2-4a75-853f-9af2719166b0","Type":"ContainerDied","Data":"c07ffc1dae0b62ba1727af6db63fe680140252d7ba6ccaba98f4d24ca0ccfc6a"} Jul 30 17:30:09.368036 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:09.368023 2550 scope.go:117] "RemoveContainer" containerID="c07ffc1dae0b62ba1727af6db63fe680140252d7ba6ccaba98f4d24ca0ccfc6a" Jul 30 17:30:10.371199 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:10.371165 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" event={"ID":"128a0597-24f2-4a75-853f-9af2719166b0","Type":"ContainerStarted","Data":"61f9dcd1a760482491f7fa56af5488e9c3a8c5524357ff3504d7259190eb5218"} Jul 30 17:30:10.371572 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:10.371485 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:30:10.372014 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:10.371996 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6899f45f4b-b7g65" Jul 30 17:30:57.666292 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:30:57.666258 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:30:57.666745 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:30:57.666372 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 17:30:57.666745 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:30:57.666423 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs podName:58726591-2751-4c7d-88c4-b916a4cdc736 nodeName:}" failed. No retries permitted until 2026-07-30 17:32:59.666409095 +0000 UTC m=+374.398923756 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs") pod "network-metrics-daemon-8nlrg" (UID: "58726591-2751-4c7d-88c4-b916a4cdc736") : secret "metrics-daemon-secret" not found Jul 30 17:31:12.328612 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.328577 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5447b65c5-h62hv"] Jul 30 17:31:12.331197 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.331182 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.336159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.336134 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zbzgx\"" Jul 30 17:31:12.336159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.336159 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 30 17:31:12.336349 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.336176 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 30 17:31:12.336349 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.336265 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 17:31:12.336349 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.336276 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 30 17:31:12.337461 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.337445 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 30 17:31:12.337546 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.337449 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 17:31:12.357177 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.357153 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5447b65c5-h62hv"] Jul 30 17:31:12.367250 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.367229 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.367352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.367279 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-stats-auth\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.367352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.367324 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.367352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.367341 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9h2j\" (UniqueName: \"kubernetes.io/projected/a5011c56-b6f0-442a-9d09-644fcba552c0-kube-api-access-g9h2j\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.367511 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.367402 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-default-certificate\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.468297 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.468263 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9h2j\" (UniqueName: \"kubernetes.io/projected/a5011c56-b6f0-442a-9d09-644fcba552c0-kube-api-access-g9h2j\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.468491 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.468354 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-default-certificate\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.468491 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.468391 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.468603 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:12.468513 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:12.968494191 +0000 UTC m=+267.701008870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : configmap references non-existent config key: service-ca.crt Jul 30 17:31:12.468603 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.468534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-stats-auth\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.468603 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.468564 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.468758 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:12.468644 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 17:31:12.468758 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:12.468687 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:12.968672003 +0000 UTC m=+267.701186665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : secret "router-metrics-certs-default" not found Jul 30 17:31:12.471045 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.471023 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-stats-auth\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.471178 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.471065 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-default-certificate\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.478600 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.478577 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9h2j\" (UniqueName: \"kubernetes.io/projected/a5011c56-b6f0-442a-9d09-644fcba552c0-kube-api-access-g9h2j\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.971669 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.971632 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.971860 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:12.971686 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:12.971860 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:12.971765 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 17:31:12.971860 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:12.971796 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:13.971772469 +0000 UTC m=+268.704287132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : configmap references non-existent config key: service-ca.crt Jul 30 17:31:12.971977 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:12.971889 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:13.971881323 +0000 UTC m=+268.704395984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : secret "router-metrics-certs-default" not found Jul 30 17:31:13.979662 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:13.979633 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:13.980075 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:13.979732 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:13.980075 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:13.979770 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 17:31:13.980075 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:13.979855 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:15.979837302 +0000 UTC m=+270.712351967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : secret "router-metrics-certs-default" not found Jul 30 17:31:13.980075 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:13.979869 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:15.979863557 +0000 UTC m=+270.712378218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : configmap references non-existent config key: service-ca.crt Jul 30 17:31:15.993485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:15.993453 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:15.993867 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:15.993503 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:15.993867 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:15.993590 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 17:31:15.993867 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:15.993615 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:19.993599168 +0000 UTC m=+274.726113829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : configmap references non-existent config key: service-ca.crt Jul 30 17:31:15.993867 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:15.993636 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:19.993629985 +0000 UTC m=+274.726144646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : secret "router-metrics-certs-default" not found Jul 30 17:31:16.470996 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.470964 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jqgd7"] Jul 30 17:31:16.473987 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.473972 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.480600 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.480576 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 17:31:16.482016 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.481995 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85bdp\"" Jul 30 17:31:16.482144 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.482040 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 17:31:16.482144 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.482046 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 17:31:16.482144 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.482043 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 17:31:16.492214 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.492195 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jqgd7"] Jul 30 17:31:16.598044 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.598016 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjsql\" (UniqueName: \"kubernetes.io/projected/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-api-access-pjsql\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.598206 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.598070 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79c050c9-18f5-4538-b6b0-adc4224c24c9-crio-socket\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.598206 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.598108 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79c050c9-18f5-4538-b6b0-adc4224c24c9-data-volume\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.598206 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.598136 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.598206 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.598183 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.699503 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699475 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.699673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699520 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjsql\" (UniqueName: \"kubernetes.io/projected/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-api-access-pjsql\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.699673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699559 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79c050c9-18f5-4538-b6b0-adc4224c24c9-crio-socket\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.699673 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:16.699593 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 17:31:16.699673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699638 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79c050c9-18f5-4538-b6b0-adc4224c24c9-data-volume\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.699673 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:16.699662 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:17.199641152 +0000 UTC m=+271.932155830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : configmap "kube-rbac-proxy" not found Jul 30 17:31:16.699673 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699662 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79c050c9-18f5-4538-b6b0-adc4224c24c9-crio-socket\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.700013 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699719 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.700013 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:16.699866 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 17:31:16.700013 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:16.699915 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:17.199899327 +0000 UTC m=+271.932413993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : secret "insights-runtime-extractor-tls" not found Jul 30 17:31:16.700013 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.699984 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79c050c9-18f5-4538-b6b0-adc4224c24c9-data-volume\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:16.713180 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:16.713163 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjsql\" (UniqueName: \"kubernetes.io/projected/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-api-access-pjsql\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:17.203414 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:17.203380 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:17.203777 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:17.203439 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:17.203777 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:17.203528 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 17:31:17.203777 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:17.203533 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 17:31:17.203777 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:17.203584 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:18.203570001 +0000 UTC m=+272.936084666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : secret "insights-runtime-extractor-tls" not found Jul 30 17:31:17.203777 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:17.203621 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:18.203603117 +0000 UTC m=+272.936117783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : configmap "kube-rbac-proxy" not found Jul 30 17:31:18.210700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:18.210647 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:18.211109 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:18.210720 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:18.211109 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:18.210786 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 17:31:18.211109 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:18.210830 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 17:31:18.211109 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:18.210861 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:20.210847121 +0000 UTC m=+274.943361784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : secret "insights-runtime-extractor-tls" not found Jul 30 17:31:18.211109 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:18.210875 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:20.210869239 +0000 UTC m=+274.943383900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : configmap "kube-rbac-proxy" not found Jul 30 17:31:18.462584 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:18.462520 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mptvx_0107a065-c0b7-496d-ad29-8d02b454374c/dns-node-resolver/0.log" Jul 30 17:31:19.261971 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:19.261946 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8hj5m_ceb77fd3-c224-4c26-aad4-923719f14703/node-ca/0.log" Jul 30 17:31:20.024958 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.024923 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:20.025171 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.024985 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:20.025171 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.025055 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 17:31:20.025171 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.025142 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:28.025127571 +0000 UTC m=+282.757642235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : secret "router-metrics-certs-default" not found Jul 30 17:31:20.025171 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.025156 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:28.025149967 +0000 UTC m=+282.757664628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : configmap references non-existent config key: service-ca.crt Jul 30 17:31:20.034615 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.034591 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rl77f"] Jul 30 17:31:20.037485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.037465 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.040582 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.040559 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 30 17:31:20.040744 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.040621 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 30 17:31:20.040744 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.040629 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 30 17:31:20.041939 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.041924 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 30 17:31:20.042053 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.041930 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-28t5m\"" Jul 30 17:31:20.049243 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.049226 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rl77f"] Jul 30 17:31:20.064134 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.064116 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5g9cd_1743b5d7-6896-4df5-9535-49be6477dc1d/serve-healthcheck-canary/0.log" Jul 30 17:31:20.125793 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.125771 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-signing-cabundle\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.125903 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.125885 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-signing-key\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.125947 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.125913 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dghz\" (UniqueName: \"kubernetes.io/projected/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-kube-api-access-9dghz\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.226785 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.226741 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dghz\" (UniqueName: \"kubernetes.io/projected/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-kube-api-access-9dghz\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.226949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.226849 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-signing-cabundle\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.226949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.226913 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:20.227020 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.226961 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:20.227020 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.226983 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-signing-key\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.227089 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.227062 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 17:31:20.227089 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.227074 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 17:31:20.227154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.227129 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:24.227114462 +0000 UTC m=+278.959629123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : configmap "kube-rbac-proxy" not found Jul 30 17:31:20.227154 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:20.227145 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:24.227138674 +0000 UTC m=+278.959653335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : secret "insights-runtime-extractor-tls" not found Jul 30 17:31:20.227551 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.227529 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-signing-cabundle\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.229356 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.229336 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-signing-key\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.236164 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.236144 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dghz\" (UniqueName: \"kubernetes.io/projected/3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0-kube-api-access-9dghz\") pod \"service-ca-658d7b7c78-rl77f\" (UID: \"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0\") " pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.346598 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.346574 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" Jul 30 17:31:20.470036 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.469925 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rl77f"] Jul 30 17:31:20.472273 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:20.472250 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3b4240_b0ae_4fcb_a5dc_18d38c329aa0.slice/crio-d1de3e82ba590dc54d58c646a51c3a781c4d1e7176b1fb7e485c015923e7516c WatchSource:0}: Error finding container d1de3e82ba590dc54d58c646a51c3a781c4d1e7176b1fb7e485c015923e7516c: Status 404 returned error can't find the container with id d1de3e82ba590dc54d58c646a51c3a781c4d1e7176b1fb7e485c015923e7516c Jul 30 17:31:20.534953 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:20.534924 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" event={"ID":"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0","Type":"ContainerStarted","Data":"d1de3e82ba590dc54d58c646a51c3a781c4d1e7176b1fb7e485c015923e7516c"} Jul 30 17:31:22.541989 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:22.541952 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" event={"ID":"3e3b4240-b0ae-4fcb-a5dc-18d38c329aa0","Type":"ContainerStarted","Data":"4c791ffd769fe5cf9b1c8722ea086bd73b36aedd2842fcf040549dcbcb0705dd"} Jul 30 17:31:22.559991 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:22.559886 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-rl77f" podStartSLOduration=0.725604204 podStartE2EDuration="2.559870296s" totalImagesPullingTime="1.834266092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:22.559286859 +0000 UTC m=+277.291801542" watchObservedRunningTime="2026-07-30 17:31:22.559870296 +0000 UTC m=+277.292385022" Jul 30 17:31:24.264021 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:24.263984 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:24.264501 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:24.264107 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 17:31:24.264501 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:24.264115 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:24.264501 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:24.264169 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:32.264150056 +0000 UTC m=+286.996664723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : configmap "kube-rbac-proxy" not found Jul 30 17:31:24.264501 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:24.264213 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 17:31:24.264501 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:24.264253 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:32.264242406 +0000 UTC m=+286.996757075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : secret "insights-runtime-extractor-tls" not found Jul 30 17:31:27.262705 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:27.262662 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" podUID="53ddc810-ac04-4499-a5b1-bc820b719759" Jul 30 17:31:27.557387 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:27.557307 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:31:28.094799 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:28.094766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:28.095009 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:28.094871 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:28.095009 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:28.094958 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle podName:a5011c56-b6f0-442a-9d09-644fcba552c0 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:44.094937194 +0000 UTC m=+298.827451868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle") pod "router-default-5447b65c5-h62hv" (UID: "a5011c56-b6f0-442a-9d09-644fcba552c0") : configmap references non-existent config key: service-ca.crt Jul 30 17:31:28.097167 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:28.097133 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5011c56-b6f0-442a-9d09-644fcba552c0-metrics-certs\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:31.217499 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.217463 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:31:31.217499 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.217508 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:31:31.218045 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.217564 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:31:31.218045 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:31.217670 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 17:31:31.218226 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:31.218075 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert podName:53ddc810-ac04-4499-a5b1-bc820b719759 nodeName:}" failed. No retries permitted until 2026-07-30 17:33:33.218048454 +0000 UTC m=+407.950563119 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-bnbzk" (UID: "53ddc810-ac04-4499-a5b1-bc820b719759") : secret "networking-console-plugin-cert" not found Jul 30 17:31:31.220924 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.220893 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"image-registry-5f58ff6cff-zx8p4\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:31:31.221319 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.221297 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0df4a595-c599-46ba-aa60-d2dc6bd50806-metrics-tls\") pod \"dns-default-tb9js\" (UID: \"0df4a595-c599-46ba-aa60-d2dc6bd50806\") " pod="openshift-dns/dns-default-tb9js" Jul 30 17:31:31.307014 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.306994 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bns8l\"" Jul 30 17:31:31.314387 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.314373 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:31:31.407709 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.407655 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tjl7j\"" Jul 30 17:31:31.414381 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.414361 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9js" Jul 30 17:31:31.445231 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.445195 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f58ff6cff-zx8p4"] Jul 30 17:31:31.449398 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:31.449368 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd405999_57f1_4712_8fb0_6d19aa07f528.slice/crio-2d675ed2a10273ae459418baf0fbc4cc969c9451d678fc8555503c047c7169b3 WatchSource:0}: Error finding container 2d675ed2a10273ae459418baf0fbc4cc969c9451d678fc8555503c047c7169b3: Status 404 returned error can't find the container with id 2d675ed2a10273ae459418baf0fbc4cc969c9451d678fc8555503c047c7169b3 Jul 30 17:31:31.553151 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.553124 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tb9js"] Jul 30 17:31:31.556758 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:31.556739 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df4a595_c599_46ba_aa60_d2dc6bd50806.slice/crio-80b54462caed9de6a846bfce8b9c260a0191a9f7ead8fd03ed97cf9f9949529f WatchSource:0}: Error finding container 80b54462caed9de6a846bfce8b9c260a0191a9f7ead8fd03ed97cf9f9949529f: Status 404 returned error can't find the container with id 80b54462caed9de6a846bfce8b9c260a0191a9f7ead8fd03ed97cf9f9949529f Jul 30 17:31:31.570867 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.570842 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb9js" event={"ID":"0df4a595-c599-46ba-aa60-d2dc6bd50806","Type":"ContainerStarted","Data":"80b54462caed9de6a846bfce8b9c260a0191a9f7ead8fd03ed97cf9f9949529f"} Jul 30 17:31:31.572441 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.572416 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" event={"ID":"fd405999-57f1-4712-8fb0-6d19aa07f528","Type":"ContainerStarted","Data":"54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7"} Jul 30 17:31:31.572527 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.572461 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" event={"ID":"fd405999-57f1-4712-8fb0-6d19aa07f528","Type":"ContainerStarted","Data":"2d675ed2a10273ae459418baf0fbc4cc969c9451d678fc8555503c047c7169b3"} Jul 30 17:31:31.572577 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.572541 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:31:31.596161 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:31.596120 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" podStartSLOduration=285.596104654 podStartE2EDuration="4m45.596104654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:31.595396334 +0000 UTC m=+286.327911017" watchObservedRunningTime="2026-07-30 17:31:31.596104654 +0000 UTC m=+286.328619336" Jul 30 17:31:32.326069 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:32.326030 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:32.326542 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:32.326093 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:32.326542 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:32.326189 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 17:31:32.326542 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:32.326256 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls podName:79c050c9-18f5-4538-b6b0-adc4224c24c9 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:48.326238008 +0000 UTC m=+303.058752673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jqgd7" (UID: "79c050c9-18f5-4538-b6b0-adc4224c24c9") : secret "insights-runtime-extractor-tls" not found Jul 30 17:31:32.326853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:32.326833 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c050c9-18f5-4538-b6b0-adc4224c24c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:33.580198 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:33.580175 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb9js" event={"ID":"0df4a595-c599-46ba-aa60-d2dc6bd50806","Type":"ContainerStarted","Data":"598085c93051f106ad6a6e771d2fe52963e6e00eb1652ed8aec0e94b5e7f0f8d"} Jul 30 17:31:34.585325 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:34.585294 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb9js" event={"ID":"0df4a595-c599-46ba-aa60-d2dc6bd50806","Type":"ContainerStarted","Data":"bc3300cd70e6b3fe3a6808b57fb7f12cd86de4a88a6a9deb83dc7200bf3987b8"} Jul 30 17:31:34.585702 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:34.585417 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tb9js" Jul 30 17:31:34.604453 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:34.604408 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tb9js" podStartSLOduration=251.731012128 podStartE2EDuration="4m13.604393635s" totalImagesPullingTime="1.873381507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:34.603348865 +0000 UTC m=+289.335863548" watchObservedRunningTime="2026-07-30 17:31:34.604393635 +0000 UTC m=+289.336908319" Jul 30 17:31:42.803397 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:31:42.803356 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8nlrg" podUID="58726591-2751-4c7d-88c4-b916a4cdc736" Jul 30 17:31:44.114372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.114328 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:44.114936 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.114917 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5011c56-b6f0-442a-9d09-644fcba552c0-service-ca-bundle\") pod \"router-default-5447b65c5-h62hv\" (UID: \"a5011c56-b6f0-442a-9d09-644fcba552c0\") " pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:44.139357 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.139326 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:44.269120 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.269074 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5447b65c5-h62hv"] Jul 30 17:31:44.271887 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:44.271863 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5011c56_b6f0_442a_9d09_644fcba552c0.slice/crio-defec65ef08630930e7f415fdd77d6670bc847ec8fc12404f470cd81c5a61915 WatchSource:0}: Error finding container defec65ef08630930e7f415fdd77d6670bc847ec8fc12404f470cd81c5a61915: Status 404 returned error can't find the container with id defec65ef08630930e7f415fdd77d6670bc847ec8fc12404f470cd81c5a61915 Jul 30 17:31:44.594446 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.594413 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tb9js" Jul 30 17:31:44.620301 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.620268 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5447b65c5-h62hv" event={"ID":"a5011c56-b6f0-442a-9d09-644fcba552c0","Type":"ContainerStarted","Data":"2e86d1bf1dbe0c6c97c5d49f96934caf9ed33de89acb556ac46575a91d5362d8"} Jul 30 17:31:44.620476 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.620317 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5447b65c5-h62hv" event={"ID":"a5011c56-b6f0-442a-9d09-644fcba552c0","Type":"ContainerStarted","Data":"defec65ef08630930e7f415fdd77d6670bc847ec8fc12404f470cd81c5a61915"} Jul 30 17:31:44.641547 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:44.641503 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5447b65c5-h62hv" podStartSLOduration=32.641488101 podStartE2EDuration="32.641488101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:44.641213754 +0000 UTC m=+299.373728438" watchObservedRunningTime="2026-07-30 17:31:44.641488101 +0000 UTC m=+299.374002785" Jul 30 17:31:45.140459 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:45.140426 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:45.142802 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:45.142778 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:45.623388 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:45.623363 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:45.624873 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:45.624849 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5447b65c5-h62hv" Jul 30 17:31:45.630184 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:45.630162 2550 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 17:31:47.567556 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.567510 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs"] Jul 30 17:31:47.569764 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.569749 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:47.572522 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.572506 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-v2pp4\"" Jul 30 17:31:47.572597 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.572506 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 30 17:31:47.580896 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.580874 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs"] Jul 30 17:31:47.640497 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.640424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85f76932-1052-4cee-b39b-b1eabeed7f84-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hzhhs\" (UID: \"85f76932-1052-4cee-b39b-b1eabeed7f84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:47.741006 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.740974 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85f76932-1052-4cee-b39b-b1eabeed7f84-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hzhhs\" (UID: \"85f76932-1052-4cee-b39b-b1eabeed7f84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:47.743474 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.743448 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85f76932-1052-4cee-b39b-b1eabeed7f84-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hzhhs\" (UID: \"85f76932-1052-4cee-b39b-b1eabeed7f84\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:47.879083 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:47.879044 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:48.005719 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.005688 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs"] Jul 30 17:31:48.008127 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:48.008098 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f76932_1052_4cee_b39b_b1eabeed7f84.slice/crio-73a2c0b9ee2ad6250731c9fd206285e902802a215ee46b6c5b9bdbcd1a927d77 WatchSource:0}: Error finding container 73a2c0b9ee2ad6250731c9fd206285e902802a215ee46b6c5b9bdbcd1a927d77: Status 404 returned error can't find the container with id 73a2c0b9ee2ad6250731c9fd206285e902802a215ee46b6c5b9bdbcd1a927d77 Jul 30 17:31:48.009926 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.009909 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 17:31:48.347054 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.347029 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:48.349391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.349368 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c050c9-18f5-4538-b6b0-adc4224c24c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jqgd7\" (UID: \"79c050c9-18f5-4538-b6b0-adc4224c24c9\") " pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:48.585382 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.585352 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85bdp\"" Jul 30 17:31:48.593731 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.593711 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jqgd7" Jul 30 17:31:48.633880 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.633843 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" event={"ID":"85f76932-1052-4cee-b39b-b1eabeed7f84","Type":"ContainerStarted","Data":"73a2c0b9ee2ad6250731c9fd206285e902802a215ee46b6c5b9bdbcd1a927d77"} Jul 30 17:31:48.720189 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:48.720162 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jqgd7"] Jul 30 17:31:48.722837 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:48.722797 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c050c9_18f5_4538_b6b0_adc4224c24c9.slice/crio-408f5a3ad34db268e51cef74eb5de96428c987ed020c642bd0fe173dbcd7f8d3 WatchSource:0}: Error finding container 408f5a3ad34db268e51cef74eb5de96428c987ed020c642bd0fe173dbcd7f8d3: Status 404 returned error can't find the container with id 408f5a3ad34db268e51cef74eb5de96428c987ed020c642bd0fe173dbcd7f8d3 Jul 30 17:31:49.641305 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:49.640849 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" event={"ID":"85f76932-1052-4cee-b39b-b1eabeed7f84","Type":"ContainerStarted","Data":"0ec4fab60a8f8efed65748218aaeaa1de6d64c4dba2605eba3bc8bebfb91e27b"} Jul 30 17:31:49.641305 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:49.641050 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:49.642540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:49.642513 2550 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/healthz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body= Jul 30 17:31:49.642663 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:49.642563 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" podUID="85f76932-1052-4cee-b39b-b1eabeed7f84" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.14:8443/healthz\": dial tcp 10.132.0.14:8443: connect: connection refused" Jul 30 17:31:49.642842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:49.642792 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqgd7" event={"ID":"79c050c9-18f5-4538-b6b0-adc4224c24c9","Type":"ContainerStarted","Data":"408f5a3ad34db268e51cef74eb5de96428c987ed020c642bd0fe173dbcd7f8d3"} Jul 30 17:31:49.660001 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:49.659924 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" podStartSLOduration=1.121467317 podStartE2EDuration="2.659907578s" totalImagesPullingTime="1.538440261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:49.658561654 +0000 UTC m=+304.391076337" watchObservedRunningTime="2026-07-30 17:31:49.659907578 +0000 UTC m=+304.392422276" Jul 30 17:31:50.651189 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:50.651160 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hzhhs" Jul 30 17:31:51.319511 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.319476 2550 patch_prober.go:28] interesting pod/image-registry-5f58ff6cff-zx8p4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 17:31:51.319703 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.319529 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 17:31:51.638387 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.638357 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-vsnmt"] Jul 30 17:31:51.653601 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.653530 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-vsnmt"] Jul 30 17:31:51.654086 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.653633 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.656669 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.656616 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 17:31:51.656853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.656693 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 17:31:51.657160 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.657015 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 30 17:31:51.657160 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.657060 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 30 17:31:51.657160 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.657077 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hf8r7\"" Jul 30 17:31:51.657374 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.657059 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 17:31:51.777453 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.777417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40bbb89d-1774-4199-8915-293a83652819-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.777641 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.777475 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40bbb89d-1774-4199-8915-293a83652819-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.777641 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.777592 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtw77\" (UniqueName: \"kubernetes.io/projected/40bbb89d-1774-4199-8915-293a83652819-kube-api-access-dtw77\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.777787 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.777649 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40bbb89d-1774-4199-8915-293a83652819-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.878105 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.878066 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40bbb89d-1774-4199-8915-293a83652819-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.878105 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.878125 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40bbb89d-1774-4199-8915-293a83652819-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.878346 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.878203 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtw77\" (UniqueName: \"kubernetes.io/projected/40bbb89d-1774-4199-8915-293a83652819-kube-api-access-dtw77\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.878346 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.878239 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40bbb89d-1774-4199-8915-293a83652819-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.879147 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.879117 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40bbb89d-1774-4199-8915-293a83652819-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.881246 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.881220 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40bbb89d-1774-4199-8915-293a83652819-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.881430 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.881387 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40bbb89d-1774-4199-8915-293a83652819-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.886831 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.886789 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtw77\" (UniqueName: \"kubernetes.io/projected/40bbb89d-1774-4199-8915-293a83652819-kube-api-access-dtw77\") pod \"prometheus-operator-84d69f794d-vsnmt\" (UID: \"40bbb89d-1774-4199-8915-293a83652819\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:51.964727 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:51.964646 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" Jul 30 17:31:52.580995 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:52.580961 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:31:53.714283 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:53.714259 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-vsnmt"] Jul 30 17:31:53.716401 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:31:53.716379 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bbb89d_1774_4199_8915_293a83652819.slice/crio-473881588fe84f397b7e2e30ef1b7265c56a64e5da8bf0f0beda6161888a97e7 WatchSource:0}: Error finding container 473881588fe84f397b7e2e30ef1b7265c56a64e5da8bf0f0beda6161888a97e7: Status 404 returned error can't find the container with id 473881588fe84f397b7e2e30ef1b7265c56a64e5da8bf0f0beda6161888a97e7 Jul 30 17:31:54.662731 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:54.662662 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" event={"ID":"40bbb89d-1774-4199-8915-293a83652819","Type":"ContainerStarted","Data":"473881588fe84f397b7e2e30ef1b7265c56a64e5da8bf0f0beda6161888a97e7"} Jul 30 17:31:54.664609 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:54.664572 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqgd7" event={"ID":"79c050c9-18f5-4538-b6b0-adc4224c24c9","Type":"ContainerStarted","Data":"35f6dffd9220a07cad837f44e93242eaca8bbccba7976269162d010059d3579b"} Jul 30 17:31:56.802653 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:56.802619 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:31:57.677678 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:57.677631 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" event={"ID":"40bbb89d-1774-4199-8915-293a83652819","Type":"ContainerStarted","Data":"5076c3bca39a3f3d3a1ec6b2aa613b6ed3cbf33717676de18d65091335e26f60"} Jul 30 17:31:57.677982 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:57.677699 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" event={"ID":"40bbb89d-1774-4199-8915-293a83652819","Type":"ContainerStarted","Data":"01cfd5c3821364f7a1e9d1217096f44aaf624c9775fd96378a9c0d899119340f"} Jul 30 17:31:57.679504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:31:57.679473 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqgd7" event={"ID":"79c050c9-18f5-4538-b6b0-adc4224c24c9","Type":"ContainerStarted","Data":"1c8cb40bc35790576cbd79ef196da7628c6b8e81c6b405c0d8babb1a413d7414"} Jul 30 17:32:00.040492 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.038284 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-vsnmt" podStartSLOduration=5.551186646 podStartE2EDuration="9.03825835s" totalImagesPullingTime="3.487071704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:57.698155263 +0000 UTC m=+312.430669947" watchObservedRunningTime="2026-07-30 17:32:00.03825835 +0000 UTC m=+314.770773033" Jul 30 17:32:00.040492 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.039167 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw"] Jul 30 17:32:00.056885 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.055758 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-q4429"] Jul 30 17:32:00.056885 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.056552 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.061745 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.061663 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.063964 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.063577 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw"] Jul 30 17:32:00.064398 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.064227 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 30 17:32:00.064506 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.064454 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 30 17:32:00.064790 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.064750 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wpwjn\"" Jul 30 17:32:00.066594 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.066506 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-q4429"] Jul 30 17:32:00.068967 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.068329 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 30 17:32:00.068967 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.068562 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sb8jd\"" Jul 30 17:32:00.068967 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.068779 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 30 17:32:00.069145 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.069018 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 30 17:32:00.069722 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.069688 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bhzcp"] Jul 30 17:32:00.082842 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.082804 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.087062 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.086410 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wfrg5\"" Jul 30 17:32:00.087062 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.086933 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 17:32:00.087062 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.086952 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 17:32:00.087062 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.087043 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 17:32:00.146593 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146563 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32532ad3-8620-4c22-b20d-1050a6c50830-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.146593 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146604 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32532ad3-8620-4c22-b20d-1050a6c50830-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.146829 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146647 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-wtmp\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.146829 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146670 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.146829 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146711 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.146829 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146766 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-sys\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.146829 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146799 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp8tx\" (UniqueName: \"kubernetes.io/projected/32532ad3-8620-4c22-b20d-1050a6c50830-kube-api-access-sp8tx\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146842 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-root\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146881 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-textfile\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146908 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-tls\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146940 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32532ad3-8620-4c22-b20d-1050a6c50830-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146967 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.146992 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.147017 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvnrf\" (UniqueName: \"kubernetes.io/projected/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-api-access-pvnrf\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.147076 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.147058 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zrck\" (UniqueName: \"kubernetes.io/projected/fe2178e3-ac80-43c0-8af8-7bd671eef724-kube-api-access-6zrck\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.147434 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.147093 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.147434 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.147129 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2178e3-ac80-43c0-8af8-7bd671eef724-metrics-client-ca\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.147434 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.147152 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-accelerators-collector-config\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.147434 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.147183 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.247548 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247509 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-sys\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.247719 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247560 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp8tx\" (UniqueName: \"kubernetes.io/projected/32532ad3-8620-4c22-b20d-1050a6c50830-kube-api-access-sp8tx\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.247719 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247635 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-sys\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.247719 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247690 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-root\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247749 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-textfile\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247783 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-tls\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247797 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-root\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247860 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32532ad3-8620-4c22-b20d-1050a6c50830-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247909 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:32:00.247925 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 30 17:32:00.247955 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247935 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.247960 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvnrf\" (UniqueName: \"kubernetes.io/projected/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-api-access-pvnrf\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:32:00.247979 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-tls podName:fe2178e3-ac80-43c0-8af8-7bd671eef724 nodeName:}" failed. No retries permitted until 2026-07-30 17:32:00.747961231 +0000 UTC m=+315.480475896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-tls") pod "node-exporter-bhzcp" (UID: "fe2178e3-ac80-43c0-8af8-7bd671eef724") : secret "node-exporter-tls" not found Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248025 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zrck\" (UniqueName: \"kubernetes.io/projected/fe2178e3-ac80-43c0-8af8-7bd671eef724-kube-api-access-6zrck\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248050 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248081 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2178e3-ac80-43c0-8af8-7bd671eef724-metrics-client-ca\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248105 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-accelerators-collector-config\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248137 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.248273 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248145 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-textfile\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.248700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248352 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32532ad3-8620-4c22-b20d-1050a6c50830-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.248700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248387 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32532ad3-8620-4c22-b20d-1050a6c50830-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.248700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248437 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-wtmp\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.248700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248463 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.248700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248489 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.248700 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248609 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.249025 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248721 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2178e3-ac80-43c0-8af8-7bd671eef724-metrics-client-ca\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.249025 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248800 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.249025 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248945 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-wtmp\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.249025 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.248960 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.249253 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.249207 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32532ad3-8620-4c22-b20d-1050a6c50830-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.249902 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.249845 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-accelerators-collector-config\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.251669 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.251647 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.251788 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.251741 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.251892 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.251873 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32532ad3-8620-4c22-b20d-1050a6c50830-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.252320 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.252297 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32532ad3-8620-4c22-b20d-1050a6c50830-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.252515 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.252497 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.260425 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.260401 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvnrf\" (UniqueName: \"kubernetes.io/projected/ad46dda8-45f4-4f7d-be68-2caf47d7cfa2-kube-api-access-pvnrf\") pod \"kube-state-metrics-56d9f944b4-q4429\" (UID: \"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.260760 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.260742 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp8tx\" (UniqueName: \"kubernetes.io/projected/32532ad3-8620-4c22-b20d-1050a6c50830-kube-api-access-sp8tx\") pod \"openshift-state-metrics-b8f49f5c6-cbcgw\" (UID: \"32532ad3-8620-4c22-b20d-1050a6c50830\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.264790 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.264765 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zrck\" (UniqueName: \"kubernetes.io/projected/fe2178e3-ac80-43c0-8af8-7bd671eef724-kube-api-access-6zrck\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.372359 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.372217 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" Jul 30 17:32:00.379781 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.379755 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" Jul 30 17:32:00.753520 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.753450 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-tls\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.755696 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.755677 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe2178e3-ac80-43c0-8af8-7bd671eef724-node-exporter-tls\") pod \"node-exporter-bhzcp\" (UID: \"fe2178e3-ac80-43c0-8af8-7bd671eef724\") " pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.961190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.961163 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw"] Jul 30 17:32:00.964193 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:32:00.964166 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32532ad3_8620_4c22_b20d_1050a6c50830.slice/crio-aa71fdedbbb4e38b7aec2105f35b3fbb35049fae2e2123de7a61dc415a424cca WatchSource:0}: Error finding container aa71fdedbbb4e38b7aec2105f35b3fbb35049fae2e2123de7a61dc415a424cca: Status 404 returned error can't find the container with id aa71fdedbbb4e38b7aec2105f35b3fbb35049fae2e2123de7a61dc415a424cca Jul 30 17:32:00.985802 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.985781 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-q4429"] Jul 30 17:32:00.992948 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:00.992931 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bhzcp" Jul 30 17:32:00.994727 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:32:00.994707 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad46dda8_45f4_4f7d_be68_2caf47d7cfa2.slice/crio-5ba32727d4b52eafdd6c4f1bceb7345a7aa78ea242ee78673fe9ea9d65d4e883 WatchSource:0}: Error finding container 5ba32727d4b52eafdd6c4f1bceb7345a7aa78ea242ee78673fe9ea9d65d4e883: Status 404 returned error can't find the container with id 5ba32727d4b52eafdd6c4f1bceb7345a7aa78ea242ee78673fe9ea9d65d4e883 Jul 30 17:32:01.008782 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:32:01.008759 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2178e3_ac80_43c0_8af8_7bd671eef724.slice/crio-a98a3b49d09fea881c37567467f07fe89383551defacf4b9a55530d839d41dc0 WatchSource:0}: Error finding container a98a3b49d09fea881c37567467f07fe89383551defacf4b9a55530d839d41dc0: Status 404 returned error can't find the container with id a98a3b49d09fea881c37567467f07fe89383551defacf4b9a55530d839d41dc0 Jul 30 17:32:01.183537 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.183501 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f58ff6cff-zx8p4"] Jul 30 17:32:01.698202 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.698150 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jqgd7" event={"ID":"79c050c9-18f5-4538-b6b0-adc4224c24c9","Type":"ContainerStarted","Data":"be09a08fcfb7bb55db4a46a52f46c87308ccf359fdc6dadeb13e0afb33d283a0"} Jul 30 17:32:01.699548 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.699520 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bhzcp" event={"ID":"fe2178e3-ac80-43c0-8af8-7bd671eef724","Type":"ContainerStarted","Data":"a98a3b49d09fea881c37567467f07fe89383551defacf4b9a55530d839d41dc0"} Jul 30 17:32:01.700714 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.700685 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" event={"ID":"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2","Type":"ContainerStarted","Data":"5ba32727d4b52eafdd6c4f1bceb7345a7aa78ea242ee78673fe9ea9d65d4e883"} Jul 30 17:32:01.702652 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.702616 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" event={"ID":"32532ad3-8620-4c22-b20d-1050a6c50830","Type":"ContainerStarted","Data":"2c5b1a82077d924084b1ce99fc37965bf9aacd295653bda12c5e0078c77faa39"} Jul 30 17:32:01.702803 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.702693 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" event={"ID":"32532ad3-8620-4c22-b20d-1050a6c50830","Type":"ContainerStarted","Data":"a7fc4fef2f03554a340553ad6f899bdcdc63cb299d3cb65ca10bcda8f7f916d4"} Jul 30 17:32:01.702803 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.702723 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" event={"ID":"32532ad3-8620-4c22-b20d-1050a6c50830","Type":"ContainerStarted","Data":"aa71fdedbbb4e38b7aec2105f35b3fbb35049fae2e2123de7a61dc415a424cca"} Jul 30 17:32:01.721875 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:01.721804 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jqgd7" podStartSLOduration=33.672978648 podStartE2EDuration="45.72178625s" totalImagesPullingTime="12.048807602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:31:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:01.719773176 +0000 UTC m=+316.452287862" watchObservedRunningTime="2026-07-30 17:32:01.72178625 +0000 UTC m=+316.454300935" Jul 30 17:32:02.707307 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:02.707234 2550 generic.go:356] "Generic (PLEG): container finished" podID="fe2178e3-ac80-43c0-8af8-7bd671eef724" containerID="f2c5460055575b014a0c69928e0cb28d4e57f78f6938573a166146958a9c0223" exitCode=0 Jul 30 17:32:02.707727 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:02.707314 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bhzcp" event={"ID":"fe2178e3-ac80-43c0-8af8-7bd671eef724","Type":"ContainerDied","Data":"f2c5460055575b014a0c69928e0cb28d4e57f78f6938573a166146958a9c0223"} Jul 30 17:32:03.717207 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.717163 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" event={"ID":"32532ad3-8620-4c22-b20d-1050a6c50830","Type":"ContainerStarted","Data":"da2476a516c6113786f1907aeff95ffb18724c5dac575d907d595f6cfbd251f4"} Jul 30 17:32:03.719167 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.719144 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bhzcp" event={"ID":"fe2178e3-ac80-43c0-8af8-7bd671eef724","Type":"ContainerStarted","Data":"574f33d41813e0eb20d604d395c63b2b6cf49f8d51203a4c8be9b5929e43e9d6"} Jul 30 17:32:03.719326 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.719184 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bhzcp" event={"ID":"fe2178e3-ac80-43c0-8af8-7bd671eef724","Type":"ContainerStarted","Data":"151ef89f865001eddde2b34a9cd7373a199609e74114431d6e6864febe3c0554"} Jul 30 17:32:03.721060 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.721040 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" event={"ID":"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2","Type":"ContainerStarted","Data":"f77c0cc4e8fb26c7d37808cb8e25fddb30b252a7045a52d12cb2ab433561cec6"} Jul 30 17:32:03.721180 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.721096 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" event={"ID":"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2","Type":"ContainerStarted","Data":"3a8faf6b82e59a6e9312f5eecd3a1db792543b772e57eedbb4dbe5ec9d1c8a12"} Jul 30 17:32:03.721180 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.721131 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" event={"ID":"ad46dda8-45f4-4f7d-be68-2caf47d7cfa2","Type":"ContainerStarted","Data":"c287164799d7c269774fee64b73f2ad5d365875668aba0150e2364292fe6040f"} Jul 30 17:32:03.741705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.741664 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbcgw" podStartSLOduration=2.115079976 podStartE2EDuration="3.74164739s" totalImagesPullingTime="1.626567414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:03.741410938 +0000 UTC m=+318.473925624" watchObservedRunningTime="2026-07-30 17:32:03.74164739 +0000 UTC m=+318.474162076" Jul 30 17:32:03.762306 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.762264 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-q4429" podStartSLOduration=2.02341238 podStartE2EDuration="3.76225065s" totalImagesPullingTime="1.73883827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:03.762118198 +0000 UTC m=+318.494632883" watchObservedRunningTime="2026-07-30 17:32:03.76225065 +0000 UTC m=+318.494765334" Jul 30 17:32:03.783186 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:03.783151 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bhzcp" podStartSLOduration=2.640016009 podStartE2EDuration="3.783138676s" totalImagesPullingTime="1.068605414s" totalInitContainerRuntime="74.517253ms" isStatefulPod=true podCreationTimestamp="2026-07-30 17:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:03.7821462 +0000 UTC m=+318.514660894" watchObservedRunningTime="2026-07-30 17:32:03.783138676 +0000 UTC m=+318.515653358" Jul 30 17:32:11.768705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.768650 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-gsj92"] Jul 30 17:32:11.772369 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.772354 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:11.775169 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.775153 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 17:32:11.775296 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.775278 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 17:32:11.775357 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.775346 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-r8p7j\"" Jul 30 17:32:11.782747 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.782727 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-gsj92"] Jul 30 17:32:11.843195 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.843168 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr98d\" (UniqueName: \"kubernetes.io/projected/ae5247db-98b7-47a7-bc64-ab7db630ee16-kube-api-access-vr98d\") pod \"downloads-7c97dc4555-gsj92\" (UID: \"ae5247db-98b7-47a7-bc64-ab7db630ee16\") " pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:11.944278 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.944249 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr98d\" (UniqueName: \"kubernetes.io/projected/ae5247db-98b7-47a7-bc64-ab7db630ee16-kube-api-access-vr98d\") pod \"downloads-7c97dc4555-gsj92\" (UID: \"ae5247db-98b7-47a7-bc64-ab7db630ee16\") " pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:11.955449 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:11.955419 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr98d\" (UniqueName: \"kubernetes.io/projected/ae5247db-98b7-47a7-bc64-ab7db630ee16-kube-api-access-vr98d\") pod \"downloads-7c97dc4555-gsj92\" (UID: \"ae5247db-98b7-47a7-bc64-ab7db630ee16\") " pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:12.081699 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:12.081620 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:12.208284 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:12.208259 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-gsj92"] Jul 30 17:32:12.210219 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:32:12.210191 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5247db_98b7_47a7_bc64_ab7db630ee16.slice/crio-dc1c0dc51f8936ba4201adf7f782ef73730852768be2541441416be699641ea3 WatchSource:0}: Error finding container dc1c0dc51f8936ba4201adf7f782ef73730852768be2541441416be699641ea3: Status 404 returned error can't find the container with id dc1c0dc51f8936ba4201adf7f782ef73730852768be2541441416be699641ea3 Jul 30 17:32:12.755985 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:12.755914 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-gsj92" event={"ID":"ae5247db-98b7-47a7-bc64-ab7db630ee16","Type":"ContainerStarted","Data":"dc1c0dc51f8936ba4201adf7f782ef73730852768be2541441416be699641ea3"} Jul 30 17:32:16.297197 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.297164 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74cd74c7f7-wfrq7"] Jul 30 17:32:16.300663 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.300636 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.304043 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.304013 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 17:32:16.304175 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.304016 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 17:32:16.304175 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.304021 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7zw78\"" Jul 30 17:32:16.304175 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.304128 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 17:32:16.305003 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.304982 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 17:32:16.305120 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.305019 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 17:32:16.309938 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.309506 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cd74c7f7-wfrq7"] Jul 30 17:32:16.382257 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.382227 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-config\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.382408 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.382264 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-oauth-config\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.382408 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.382331 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tz5p\" (UniqueName: \"kubernetes.io/projected/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-kube-api-access-2tz5p\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.382408 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.382354 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-service-ca\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.382408 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.382391 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-serving-cert\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.382590 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.382411 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-oauth-serving-cert\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.483393 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.483363 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-config\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.483540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.483397 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-oauth-config\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.483540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.483438 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tz5p\" (UniqueName: \"kubernetes.io/projected/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-kube-api-access-2tz5p\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.483540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.483463 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-service-ca\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.483540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.483490 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-serving-cert\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.483540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.483511 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-oauth-serving-cert\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.484193 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.484149 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-config\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.484549 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.484509 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-service-ca\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.484670 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.484547 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-oauth-serving-cert\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.486246 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.486228 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-oauth-config\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.486585 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.486563 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-serving-cert\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.492414 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.492392 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tz5p\" (UniqueName: \"kubernetes.io/projected/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-kube-api-access-2tz5p\") pod \"console-74cd74c7f7-wfrq7\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.612736 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.612702 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:16.764962 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:16.764938 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cd74c7f7-wfrq7"] Jul 30 17:32:16.768000 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:32:16.767972 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e19daf3_c28b_4d67_a3f1_b9409c5d9385.slice/crio-31803641ac5499c6b91ff682ffc6fab3b506eb029a34b9cd30b2f3ca22858d08 WatchSource:0}: Error finding container 31803641ac5499c6b91ff682ffc6fab3b506eb029a34b9cd30b2f3ca22858d08: Status 404 returned error can't find the container with id 31803641ac5499c6b91ff682ffc6fab3b506eb029a34b9cd30b2f3ca22858d08 Jul 30 17:32:17.777219 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:17.777109 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cd74c7f7-wfrq7" event={"ID":"5e19daf3-c28b-4d67-a3f1-b9409c5d9385","Type":"ContainerStarted","Data":"31803641ac5499c6b91ff682ffc6fab3b506eb029a34b9cd30b2f3ca22858d08"} Jul 30 17:32:20.791896 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:20.791852 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cd74c7f7-wfrq7" event={"ID":"5e19daf3-c28b-4d67-a3f1-b9409c5d9385","Type":"ContainerStarted","Data":"bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50"} Jul 30 17:32:20.813328 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:20.813275 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74cd74c7f7-wfrq7" podStartSLOduration=1.32546322 podStartE2EDuration="4.813255211s" totalImagesPullingTime="3.487791991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:20.810919 +0000 UTC m=+335.543433683" watchObservedRunningTime="2026-07-30 17:32:20.813255211 +0000 UTC m=+335.545769896" Jul 30 17:32:26.202506 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:26.202464 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" containerName="registry" containerID="cri-o://54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7" gracePeriod=30 Jul 30 17:32:26.613091 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:26.613049 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:26.613284 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:26.613124 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:26.618773 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:26.618749 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:26.820927 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:26.820897 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:32:29.699973 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.699943 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:32:29.803987 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.803964 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-installation-pull-secrets\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804093 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804000 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804093 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804075 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-trusted-ca\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804099 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-bound-sa-token\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804126 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-image-registry-private-configuration\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804160 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpvn5\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-kube-api-access-mpvn5\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804209 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-certificates\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804352 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804252 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd405999-57f1-4712-8fb0-6d19aa07f528-ca-trust-extracted\") pod \"fd405999-57f1-4712-8fb0-6d19aa07f528\" (UID: \"fd405999-57f1-4712-8fb0-6d19aa07f528\") " Jul 30 17:32:29.804571 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804538 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:32:29.805046 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.804963 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:32:29.807014 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.806990 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:32:29.807256 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.807229 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:32:29.807600 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.807576 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:32:29.807600 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.807577 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:32:29.808258 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.808231 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-kube-api-access-mpvn5" (OuterVolumeSpecName: "kube-api-access-mpvn5") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "kube-api-access-mpvn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:32:29.813290 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.813269 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd405999-57f1-4712-8fb0-6d19aa07f528-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fd405999-57f1-4712-8fb0-6d19aa07f528" (UID: "fd405999-57f1-4712-8fb0-6d19aa07f528"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:32:29.828988 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.828967 2550 generic.go:356] "Generic (PLEG): container finished" podID="fd405999-57f1-4712-8fb0-6d19aa07f528" containerID="54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7" exitCode=0 Jul 30 17:32:29.829071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.829009 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" Jul 30 17:32:29.829071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.829013 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" event={"ID":"fd405999-57f1-4712-8fb0-6d19aa07f528","Type":"ContainerDied","Data":"54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7"} Jul 30 17:32:29.829071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.829049 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f58ff6cff-zx8p4" event={"ID":"fd405999-57f1-4712-8fb0-6d19aa07f528","Type":"ContainerDied","Data":"2d675ed2a10273ae459418baf0fbc4cc969c9451d678fc8555503c047c7169b3"} Jul 30 17:32:29.829071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.829066 2550 scope.go:117] "RemoveContainer" containerID="54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7" Jul 30 17:32:29.838036 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.838016 2550 scope.go:117] "RemoveContainer" containerID="54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7" Jul 30 17:32:29.838313 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:32:29.838292 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7\": container with ID starting with 54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7 not found: ID does not exist" containerID="54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7" Jul 30 17:32:29.838395 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.838320 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7"} err="failed to get container status \"54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7\": rpc error: code = NotFound desc = could not find container \"54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7\": container with ID starting with 54d16d22e6f58c81cc1b8e2772a37cbf2f3e5c62cec19b3e361ea3a2a32fb5d7 not found: ID does not exist" Jul 30 17:32:29.850416 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.850395 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f58ff6cff-zx8p4"] Jul 30 17:32:29.856871 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.854967 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f58ff6cff-zx8p4"] Jul 30 17:32:29.905570 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905545 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-certificates\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905570 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905567 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd405999-57f1-4712-8fb0-6d19aa07f528-ca-trust-extracted\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905767 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905654 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-installation-pull-secrets\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905767 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905756 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-registry-tls\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905889 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905784 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd405999-57f1-4712-8fb0-6d19aa07f528-trusted-ca\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905889 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905800 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-bound-sa-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905889 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905838 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd405999-57f1-4712-8fb0-6d19aa07f528-image-registry-private-configuration\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:29.905889 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:29.905855 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpvn5\" (UniqueName: \"kubernetes.io/projected/fd405999-57f1-4712-8fb0-6d19aa07f528-kube-api-access-mpvn5\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:32:30.836140 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:30.836056 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-gsj92" event={"ID":"ae5247db-98b7-47a7-bc64-ab7db630ee16","Type":"ContainerStarted","Data":"02cfb4a28bdf21a0cba72695865d478653f467f9c1f7268cbf50a3da14c99bc9"} Jul 30 17:32:30.836606 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:30.836360 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:30.851068 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:30.851042 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-gsj92" Jul 30 17:32:30.862973 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:30.862932 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-gsj92" podStartSLOduration=2.331287378 podStartE2EDuration="19.862917865s" totalImagesPullingTime="17.531630487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:30.861374348 +0000 UTC m=+345.593889043" watchObservedRunningTime="2026-07-30 17:32:30.862917865 +0000 UTC m=+345.595432549" Jul 30 17:32:31.808098 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:31.808057 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" path="/var/lib/kubelet/pods/fd405999-57f1-4712-8fb0-6d19aa07f528/volumes" Jul 30 17:32:36.404234 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:36.404202 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cd74c7f7-wfrq7"] Jul 30 17:32:37.338908 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:37.338782 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5447b65c5-h62hv_a5011c56-b6f0-442a-9d09-644fcba552c0/router/0.log" Jul 30 17:32:37.345438 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:37.345410 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5g9cd_1743b5d7-6896-4df5-9535-49be6477dc1d/serve-healthcheck-canary/0.log" Jul 30 17:32:59.741677 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:59.741644 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:32:59.744097 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:59.744073 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58726591-2751-4c7d-88c4-b916a4cdc736-metrics-certs\") pod \"network-metrics-daemon-8nlrg\" (UID: \"58726591-2751-4c7d-88c4-b916a4cdc736\") " pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:32:59.806367 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:59.806341 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjwv8\"" Jul 30 17:32:59.813710 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:59.813690 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8nlrg" Jul 30 17:32:59.996632 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:32:59.996607 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8nlrg"] Jul 30 17:32:59.998954 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:32:59.998930 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58726591_2751_4c7d_88c4_b916a4cdc736.slice/crio-fe5bedb608abc33150643059dc5f5ddde104223b04ab32792f8cfb3ca1e309f1 WatchSource:0}: Error finding container fe5bedb608abc33150643059dc5f5ddde104223b04ab32792f8cfb3ca1e309f1: Status 404 returned error can't find the container with id fe5bedb608abc33150643059dc5f5ddde104223b04ab32792f8cfb3ca1e309f1 Jul 30 17:33:00.952241 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:00.952201 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8nlrg" event={"ID":"58726591-2751-4c7d-88c4-b916a4cdc736","Type":"ContainerStarted","Data":"fe5bedb608abc33150643059dc5f5ddde104223b04ab32792f8cfb3ca1e309f1"} Jul 30 17:33:01.427229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.427131 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74cd74c7f7-wfrq7" podUID="5e19daf3-c28b-4d67-a3f1-b9409c5d9385" containerName="console" containerID="cri-o://bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50" gracePeriod=15 Jul 30 17:33:01.729353 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.729329 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:33:01.758824 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.758779 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-serving-cert\") pod \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " Jul 30 17:33:01.758957 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.758854 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-config\") pod \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " Jul 30 17:33:01.758957 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.758895 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-oauth-config\") pod \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " Jul 30 17:33:01.758957 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.758938 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-oauth-serving-cert\") pod \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " Jul 30 17:33:01.759118 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.758969 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tz5p\" (UniqueName: \"kubernetes.io/projected/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-kube-api-access-2tz5p\") pod \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " Jul 30 17:33:01.759118 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.758995 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-service-ca\") pod \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\" (UID: \"5e19daf3-c28b-4d67-a3f1-b9409c5d9385\") " Jul 30 17:33:01.759237 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.759206 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-config" (OuterVolumeSpecName: "console-config") pod "5e19daf3-c28b-4d67-a3f1-b9409c5d9385" (UID: "5e19daf3-c28b-4d67-a3f1-b9409c5d9385"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:33:01.759624 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.759471 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e19daf3-c28b-4d67-a3f1-b9409c5d9385" (UID: "5e19daf3-c28b-4d67-a3f1-b9409c5d9385"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:33:01.759624 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.759482 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e19daf3-c28b-4d67-a3f1-b9409c5d9385" (UID: "5e19daf3-c28b-4d67-a3f1-b9409c5d9385"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:33:01.761164 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.761121 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e19daf3-c28b-4d67-a3f1-b9409c5d9385" (UID: "5e19daf3-c28b-4d67-a3f1-b9409c5d9385"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:33:01.761327 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.761206 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e19daf3-c28b-4d67-a3f1-b9409c5d9385" (UID: "5e19daf3-c28b-4d67-a3f1-b9409c5d9385"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:33:01.761327 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.761216 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-kube-api-access-2tz5p" (OuterVolumeSpecName: "kube-api-access-2tz5p") pod "5e19daf3-c28b-4d67-a3f1-b9409c5d9385" (UID: "5e19daf3-c28b-4d67-a3f1-b9409c5d9385"). InnerVolumeSpecName "kube-api-access-2tz5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:33:01.860391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.860364 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tz5p\" (UniqueName: \"kubernetes.io/projected/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-kube-api-access-2tz5p\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:33:01.860391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.860386 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-service-ca\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:33:01.860391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.860395 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-serving-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:33:01.860572 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.860405 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-config\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:33:01.860572 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.860414 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-console-oauth-config\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:33:01.860572 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.860435 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e19daf3-c28b-4d67-a3f1-b9409c5d9385-oauth-serving-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:33:01.958961 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.958895 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e19daf3-c28b-4d67-a3f1-b9409c5d9385" containerID="bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50" exitCode=0 Jul 30 17:33:01.958961 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.958942 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cd74c7f7-wfrq7" Jul 30 17:33:01.959384 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.958977 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cd74c7f7-wfrq7" event={"ID":"5e19daf3-c28b-4d67-a3f1-b9409c5d9385","Type":"ContainerDied","Data":"bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50"} Jul 30 17:33:01.959384 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.959035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cd74c7f7-wfrq7" event={"ID":"5e19daf3-c28b-4d67-a3f1-b9409c5d9385","Type":"ContainerDied","Data":"31803641ac5499c6b91ff682ffc6fab3b506eb029a34b9cd30b2f3ca22858d08"} Jul 30 17:33:01.959384 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.959062 2550 scope.go:117] "RemoveContainer" containerID="bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50" Jul 30 17:33:01.960967 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.960938 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8nlrg" event={"ID":"58726591-2751-4c7d-88c4-b916a4cdc736","Type":"ContainerStarted","Data":"30ac3efcdc18bc7afd668373b5a66435331600f5f1a61a450f6eccc2bc3eb8db"} Jul 30 17:33:01.961063 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.960992 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8nlrg" event={"ID":"58726591-2751-4c7d-88c4-b916a4cdc736","Type":"ContainerStarted","Data":"a64f576b87fe3d0ffbadca39d5716abdf6f185e143147130be6b5bac87a455e7"} Jul 30 17:33:01.967088 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.967073 2550 scope.go:117] "RemoveContainer" containerID="bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50" Jul 30 17:33:01.967324 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:33:01.967308 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50\": container with ID starting with bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50 not found: ID does not exist" containerID="bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50" Jul 30 17:33:01.967373 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.967331 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50"} err="failed to get container status \"bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50\": rpc error: code = NotFound desc = could not find container \"bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50\": container with ID starting with bfafdbc37d75a97f0eb8dfe13e03a340c8e60c822d1e312da98cc0446f69ce50 not found: ID does not exist" Jul 30 17:33:01.982478 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.982443 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8nlrg" podStartSLOduration=375.614385896 podStartE2EDuration="6m16.982431641s" totalImagesPullingTime="1.368045745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:01.981984656 +0000 UTC m=+376.714499340" watchObservedRunningTime="2026-07-30 17:33:01.982431641 +0000 UTC m=+376.714946324" Jul 30 17:33:01.996689 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:01.996662 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cd74c7f7-wfrq7"] Jul 30 17:33:02.000436 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:02.000417 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74cd74c7f7-wfrq7"] Jul 30 17:33:03.807982 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:03.807953 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e19daf3-c28b-4d67-a3f1-b9409c5d9385" path="/var/lib/kubelet/pods/5e19daf3-c28b-4d67-a3f1-b9409c5d9385/volumes" Jul 30 17:33:28.520872 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.520799 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65b887df54-dllnz"] Jul 30 17:33:28.521372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.521239 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" containerName="registry" Jul 30 17:33:28.521372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.521259 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" containerName="registry" Jul 30 17:33:28.521372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.521286 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e19daf3-c28b-4d67-a3f1-b9409c5d9385" containerName="console" Jul 30 17:33:28.521372 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.521296 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e19daf3-c28b-4d67-a3f1-b9409c5d9385" containerName="console" Jul 30 17:33:28.521522 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.521390 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e19daf3-c28b-4d67-a3f1-b9409c5d9385" containerName="console" Jul 30 17:33:28.521522 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.521403 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd405999-57f1-4712-8fb0-6d19aa07f528" containerName="registry" Jul 30 17:33:28.524169 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.523562 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.528758 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.528740 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 17:33:28.530478 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.530297 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 17:33:28.531169 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.531149 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 17:33:28.534542 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.534520 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 17:33:28.539287 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.539268 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 17:33:28.546920 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.546901 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7zw78\"" Jul 30 17:33:28.547044 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.547021 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b887df54-dllnz"] Jul 30 17:33:28.568358 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568327 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-config\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.568479 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568379 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngr78\" (UniqueName: \"kubernetes.io/projected/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-kube-api-access-ngr78\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.568479 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568414 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-oauth-config\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.568479 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568461 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-serving-cert\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.568630 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568486 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-trusted-ca-bundle\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.568630 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568513 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-oauth-serving-cert\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.568630 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.568552 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-service-ca\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.584109 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.584081 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 30 17:33:28.669553 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669516 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-serving-cert\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.669553 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669553 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-trusted-ca-bundle\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.669804 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669582 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-oauth-serving-cert\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.669804 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669627 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-service-ca\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.669804 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669663 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-config\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.669804 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669700 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngr78\" (UniqueName: \"kubernetes.io/projected/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-kube-api-access-ngr78\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.669804 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.669741 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-oauth-config\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.670518 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.670478 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-oauth-serving-cert\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.670655 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.670513 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-config\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.670655 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.670542 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-trusted-ca-bundle\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.670655 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.670575 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-service-ca\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.672680 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.672661 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-serving-cert\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.672778 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.672761 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-oauth-config\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.678241 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.678186 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngr78\" (UniqueName: \"kubernetes.io/projected/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-kube-api-access-ngr78\") pod \"console-65b887df54-dllnz\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.833959 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.833865 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:28.986930 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:28.986869 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b887df54-dllnz"] Jul 30 17:33:28.991476 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:33:28.991435 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1097135_6f7b_4bcb_a1ce_75ab53fc4103.slice/crio-93a21b69cc5de780fcb37b954eaf08064ddd44d73ba171bd4aa40b0764af48f1 WatchSource:0}: Error finding container 93a21b69cc5de780fcb37b954eaf08064ddd44d73ba171bd4aa40b0764af48f1: Status 404 returned error can't find the container with id 93a21b69cc5de780fcb37b954eaf08064ddd44d73ba171bd4aa40b0764af48f1 Jul 30 17:33:29.063530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:29.063495 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b887df54-dllnz" event={"ID":"b1097135-6f7b-4bcb-a1ce-75ab53fc4103","Type":"ContainerStarted","Data":"7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794"} Jul 30 17:33:29.063656 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:29.063566 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b887df54-dllnz" event={"ID":"b1097135-6f7b-4bcb-a1ce-75ab53fc4103","Type":"ContainerStarted","Data":"93a21b69cc5de780fcb37b954eaf08064ddd44d73ba171bd4aa40b0764af48f1"} Jul 30 17:33:29.083340 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:29.083289 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65b887df54-dllnz" podStartSLOduration=1.083272592 podStartE2EDuration="1.083272592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:29.082312407 +0000 UTC m=+403.814827090" watchObservedRunningTime="2026-07-30 17:33:29.083272592 +0000 UTC m=+403.815787275" Jul 30 17:33:30.558772 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:33:30.558731 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" podUID="53ddc810-ac04-4499-a5b1-bc820b719759" Jul 30 17:33:31.070146 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:31.070116 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:33:33.305832 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:33.305767 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:33:33.308313 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:33.308295 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53ddc810-ac04-4499-a5b1-bc820b719759-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-bnbzk\" (UID: \"53ddc810-ac04-4499-a5b1-bc820b719759\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:33:33.473724 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:33.473694 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mnxwp\"" Jul 30 17:33:33.481115 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:33.481096 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" Jul 30 17:33:33.610606 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:33.610577 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk"] Jul 30 17:33:33.613363 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:33:33.613339 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ddc810_ac04_4499_a5b1_bc820b719759.slice/crio-993eeff8c5da57e30edc7d8b4e3709a7644b2500c97915924132f6f7dd92302c WatchSource:0}: Error finding container 993eeff8c5da57e30edc7d8b4e3709a7644b2500c97915924132f6f7dd92302c: Status 404 returned error can't find the container with id 993eeff8c5da57e30edc7d8b4e3709a7644b2500c97915924132f6f7dd92302c Jul 30 17:33:34.081107 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:34.081066 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" event={"ID":"53ddc810-ac04-4499-a5b1-bc820b719759","Type":"ContainerStarted","Data":"993eeff8c5da57e30edc7d8b4e3709a7644b2500c97915924132f6f7dd92302c"} Jul 30 17:33:35.086197 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:35.086098 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" event={"ID":"53ddc810-ac04-4499-a5b1-bc820b719759","Type":"ContainerStarted","Data":"40ebb509fe32e34e5ce40d6986ccf89bfa75be83f7a4ff6b6b8d24e24636dcf2"} Jul 30 17:33:35.103646 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:35.103579 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-bnbzk" podStartSLOduration=389.891996776 podStartE2EDuration="6m31.103558351s" totalImagesPullingTime="1.211561575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:35.102673861 +0000 UTC m=+409.835188545" watchObservedRunningTime="2026-07-30 17:33:35.103558351 +0000 UTC m=+409.836073041" Jul 30 17:33:38.834787 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:38.834754 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:38.835169 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:38.834823 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:38.839443 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:38.839424 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:33:39.104715 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:33:39.104691 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:34:37.484113 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.484073 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bfb8fbbfd-dvrqx"] Jul 30 17:34:37.486617 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.486596 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.498880 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.498846 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfb8fbbfd-dvrqx"] Jul 30 17:34:37.568881 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.568856 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-oauth-serving-cert\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.569012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.568897 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-serving-cert\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.569012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.568916 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-config\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.569012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.568930 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-trusted-ca-bundle\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.569012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.568974 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-oauth-config\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.569185 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.569019 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-service-ca\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.569185 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.569054 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxs8\" (UniqueName: \"kubernetes.io/projected/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-kube-api-access-6jxs8\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670127 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670099 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-service-ca\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670262 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670132 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxs8\" (UniqueName: \"kubernetes.io/projected/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-kube-api-access-6jxs8\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670262 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670199 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-oauth-serving-cert\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670262 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670234 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-serving-cert\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670414 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670261 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-config\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670414 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670285 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-trusted-ca-bundle\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670414 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670315 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-oauth-config\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.670907 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670877 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-service-ca\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.671176 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670940 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-config\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.671176 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.670987 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-oauth-serving-cert\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.671176 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.671105 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-trusted-ca-bundle\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.672788 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.672762 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-oauth-config\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.672924 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.672906 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-serving-cert\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.679841 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.679802 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxs8\" (UniqueName: \"kubernetes.io/projected/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-kube-api-access-6jxs8\") pod \"console-7bfb8fbbfd-dvrqx\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.795269 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.795214 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:37.921787 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:37.921659 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfb8fbbfd-dvrqx"] Jul 30 17:34:37.924430 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:34:37.924408 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8657402_f4d7_47f8_8a35_af4a8b7e0c70.slice/crio-207f3d88e70b1b018916c9244b88ec9ad4d2fd36edd4219e8adb279f98e59e60 WatchSource:0}: Error finding container 207f3d88e70b1b018916c9244b88ec9ad4d2fd36edd4219e8adb279f98e59e60: Status 404 returned error can't find the container with id 207f3d88e70b1b018916c9244b88ec9ad4d2fd36edd4219e8adb279f98e59e60 Jul 30 17:34:38.315551 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:38.315518 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfb8fbbfd-dvrqx" event={"ID":"b8657402-f4d7-47f8-8a35-af4a8b7e0c70","Type":"ContainerStarted","Data":"72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d"} Jul 30 17:34:38.315712 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:38.315565 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfb8fbbfd-dvrqx" event={"ID":"b8657402-f4d7-47f8-8a35-af4a8b7e0c70","Type":"ContainerStarted","Data":"207f3d88e70b1b018916c9244b88ec9ad4d2fd36edd4219e8adb279f98e59e60"} Jul 30 17:34:38.334238 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:38.334194 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bfb8fbbfd-dvrqx" podStartSLOduration=1.334179448 podStartE2EDuration="1.334179448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:34:38.33319778 +0000 UTC m=+473.065712461" watchObservedRunningTime="2026-07-30 17:34:38.334179448 +0000 UTC m=+473.066694131" Jul 30 17:34:47.795647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:47.795608 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:47.796238 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:47.795690 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:47.800001 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:47.799984 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:48.354024 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:48.353996 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:34:48.437546 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:34:48.437513 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b887df54-dllnz"] Jul 30 17:35:13.456640 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.456602 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65b887df54-dllnz" podUID="b1097135-6f7b-4bcb-a1ce-75ab53fc4103" containerName="console" containerID="cri-o://7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794" gracePeriod=15 Jul 30 17:35:13.695392 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.695370 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:35:13.849415 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849388 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-config\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849564 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849435 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-oauth-config\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849564 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849498 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngr78\" (UniqueName: \"kubernetes.io/projected/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-kube-api-access-ngr78\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849564 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849532 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-oauth-serving-cert\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849564 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849559 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-service-ca\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849769 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849582 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-trusted-ca-bundle\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849769 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849619 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-serving-cert\") pod \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\" (UID: \"b1097135-6f7b-4bcb-a1ce-75ab53fc4103\") " Jul 30 17:35:13.849951 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.849909 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-config" (OuterVolumeSpecName: "console-config") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:35:13.850504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.850473 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-service-ca" (OuterVolumeSpecName: "service-ca") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:35:13.850504 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.850485 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:35:13.850646 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.850564 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:35:13.856376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.852697 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:35:13.856492 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.856415 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:35:13.856492 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.856455 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-kube-api-access-ngr78" (OuterVolumeSpecName: "kube-api-access-ngr78") pod "b1097135-6f7b-4bcb-a1ce-75ab53fc4103" (UID: "b1097135-6f7b-4bcb-a1ce-75ab53fc4103"). InnerVolumeSpecName "kube-api-access-ngr78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:35:13.951071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951040 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-oauth-config\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:13.951071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951069 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngr78\" (UniqueName: \"kubernetes.io/projected/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-kube-api-access-ngr78\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:13.951249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951079 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-oauth-serving-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:13.951249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951088 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-service-ca\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:13.951249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951096 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-trusted-ca-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:13.951249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951105 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-serving-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:13.951249 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:13.951113 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1097135-6f7b-4bcb-a1ce-75ab53fc4103-console-config\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:14.448221 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.448187 2550 generic.go:356] "Generic (PLEG): container finished" podID="b1097135-6f7b-4bcb-a1ce-75ab53fc4103" containerID="7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794" exitCode=0 Jul 30 17:35:14.448376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.448242 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b887df54-dllnz" Jul 30 17:35:14.448376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.448265 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b887df54-dllnz" event={"ID":"b1097135-6f7b-4bcb-a1ce-75ab53fc4103","Type":"ContainerDied","Data":"7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794"} Jul 30 17:35:14.448376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.448340 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b887df54-dllnz" event={"ID":"b1097135-6f7b-4bcb-a1ce-75ab53fc4103","Type":"ContainerDied","Data":"93a21b69cc5de780fcb37b954eaf08064ddd44d73ba171bd4aa40b0764af48f1"} Jul 30 17:35:14.448376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.448364 2550 scope.go:117] "RemoveContainer" containerID="7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794" Jul 30 17:35:14.456595 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.456582 2550 scope.go:117] "RemoveContainer" containerID="7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794" Jul 30 17:35:14.456845 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:35:14.456802 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794\": container with ID starting with 7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794 not found: ID does not exist" containerID="7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794" Jul 30 17:35:14.457148 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.456850 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794"} err="failed to get container status \"7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794\": rpc error: code = NotFound desc = could not find container \"7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794\": container with ID starting with 7ca47272e572f99225e734138fc6e9d76fb77ea0dcd712ba63854eb425bd1794 not found: ID does not exist" Jul 30 17:35:14.469797 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.469775 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b887df54-dllnz"] Jul 30 17:35:14.473852 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:14.473831 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65b887df54-dllnz"] Jul 30 17:35:15.807015 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:15.806988 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1097135-6f7b-4bcb-a1ce-75ab53fc4103" path="/var/lib/kubelet/pods/b1097135-6f7b-4bcb-a1ce-75ab53fc4103/volumes" Jul 30 17:35:35.195861 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.195827 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg"] Jul 30 17:35:35.196255 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.196110 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1097135-6f7b-4bcb-a1ce-75ab53fc4103" containerName="console" Jul 30 17:35:35.196255 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.196120 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1097135-6f7b-4bcb-a1ce-75ab53fc4103" containerName="console" Jul 30 17:35:35.196255 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.196170 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1097135-6f7b-4bcb-a1ce-75ab53fc4103" containerName="console" Jul 30 17:35:35.199031 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.199013 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.202325 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.202310 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bj8qt\"" Jul 30 17:35:35.202604 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.202590 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 17:35:35.203535 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.203521 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 17:35:35.206294 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.206276 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.206391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.206303 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.206457 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.206384 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-959q8\" (UniqueName: \"kubernetes.io/projected/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-kube-api-access-959q8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.209603 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.209586 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg"] Jul 30 17:35:35.306668 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.306646 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-959q8\" (UniqueName: \"kubernetes.io/projected/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-kube-api-access-959q8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.306791 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.306691 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.306791 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.306708 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.307075 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.307057 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.307116 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.307077 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.315995 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.315977 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-959q8\" (UniqueName: \"kubernetes.io/projected/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-kube-api-access-959q8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.509031 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.508946 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:35.638577 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:35.638545 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg"] Jul 30 17:35:35.641208 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:35:35.641181 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde29ae1d_bea6_477d_b6a1_61f2cfa6a25f.slice/crio-e00e86a92102ea049a41e2223747ac1fa785262f09b707d8afd5a157c459293b WatchSource:0}: Error finding container e00e86a92102ea049a41e2223747ac1fa785262f09b707d8afd5a157c459293b: Status 404 returned error can't find the container with id e00e86a92102ea049a41e2223747ac1fa785262f09b707d8afd5a157c459293b Jul 30 17:35:36.532090 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:36.532049 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" event={"ID":"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f","Type":"ContainerStarted","Data":"e00e86a92102ea049a41e2223747ac1fa785262f09b707d8afd5a157c459293b"} Jul 30 17:35:42.558566 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:42.558475 2550 generic.go:356] "Generic (PLEG): container finished" podID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerID="6fe1db94daa1d85ff75af849956259fcd2aeda0a2c422067b2ff6087a460f387" exitCode=0 Jul 30 17:35:42.558566 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:42.558542 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" event={"ID":"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f","Type":"ContainerDied","Data":"6fe1db94daa1d85ff75af849956259fcd2aeda0a2c422067b2ff6087a460f387"} Jul 30 17:35:45.571554 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:45.571522 2550 generic.go:356] "Generic (PLEG): container finished" podID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerID="c2e0674cac7266cf74cc609c2ed20103096dfc262f7fee432c0eb4d7d0576317" exitCode=0 Jul 30 17:35:45.571881 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:45.571575 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" event={"ID":"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f","Type":"ContainerDied","Data":"c2e0674cac7266cf74cc609c2ed20103096dfc262f7fee432c0eb4d7d0576317"} Jul 30 17:35:58.630650 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:58.630613 2550 generic.go:356] "Generic (PLEG): container finished" podID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerID="869388df9b3343de99a5a960d2646693ff49f0901c6868536b33f5c7dd1d5fed" exitCode=0 Jul 30 17:35:58.631103 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:58.630684 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" event={"ID":"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f","Type":"ContainerDied","Data":"869388df9b3343de99a5a960d2646693ff49f0901c6868536b33f5c7dd1d5fed"} Jul 30 17:35:59.755635 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.755615 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:35:59.812934 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.812912 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-bundle\") pod \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " Jul 30 17:35:59.813044 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.812961 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-959q8\" (UniqueName: \"kubernetes.io/projected/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-kube-api-access-959q8\") pod \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " Jul 30 17:35:59.813044 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.813015 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-util\") pod \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\" (UID: \"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f\") " Jul 30 17:35:59.813651 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.813627 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-bundle" (OuterVolumeSpecName: "bundle") pod "de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" (UID: "de29ae1d-bea6-477d-b6a1-61f2cfa6a25f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:35:59.815239 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.815214 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-kube-api-access-959q8" (OuterVolumeSpecName: "kube-api-access-959q8") pod "de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" (UID: "de29ae1d-bea6-477d-b6a1-61f2cfa6a25f"). InnerVolumeSpecName "kube-api-access-959q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:35:59.817614 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.817591 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-util" (OuterVolumeSpecName: "util") pod "de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" (UID: "de29ae1d-bea6-477d-b6a1-61f2cfa6a25f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:35:59.913730 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.913652 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-959q8\" (UniqueName: \"kubernetes.io/projected/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-kube-api-access-959q8\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:59.913730 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.913680 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:35:59.913730 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:35:59.913694 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de29ae1d-bea6-477d-b6a1-61f2cfa6a25f-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:00.640913 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:00.640889 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" Jul 30 17:36:00.641087 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:00.640882 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540562cg" event={"ID":"de29ae1d-bea6-477d-b6a1-61f2cfa6a25f","Type":"ContainerDied","Data":"e00e86a92102ea049a41e2223747ac1fa785262f09b707d8afd5a157c459293b"} Jul 30 17:36:00.641087 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:00.640999 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e00e86a92102ea049a41e2223747ac1fa785262f09b707d8afd5a157c459293b" Jul 30 17:36:08.391523 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391490 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8"] Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391771 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="extract" Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391782 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="extract" Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391791 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="pull" Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391796 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="pull" Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391804 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="util" Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391823 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="util" Jul 30 17:36:08.391949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.391889 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="de29ae1d-bea6-477d-b6a1-61f2cfa6a25f" containerName="extract" Jul 30 17:36:08.394283 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.394269 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.397371 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.397354 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 30 17:36:08.397571 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.397558 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mhrwv\"" Jul 30 17:36:08.397612 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.397588 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 30 17:36:08.408530 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.408507 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8"] Jul 30 17:36:08.480869 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.480838 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv67v\" (UniqueName: \"kubernetes.io/projected/fa7cb471-c256-411f-bf96-8d98b79e596d-kube-api-access-dv67v\") pod \"cert-manager-operator-controller-manager-55c444bf86-s4bd8\" (UID: \"fa7cb471-c256-411f-bf96-8d98b79e596d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.480967 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.480883 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa7cb471-c256-411f-bf96-8d98b79e596d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s4bd8\" (UID: \"fa7cb471-c256-411f-bf96-8d98b79e596d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.582053 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.582017 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv67v\" (UniqueName: \"kubernetes.io/projected/fa7cb471-c256-411f-bf96-8d98b79e596d-kube-api-access-dv67v\") pod \"cert-manager-operator-controller-manager-55c444bf86-s4bd8\" (UID: \"fa7cb471-c256-411f-bf96-8d98b79e596d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.582165 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.582073 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa7cb471-c256-411f-bf96-8d98b79e596d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s4bd8\" (UID: \"fa7cb471-c256-411f-bf96-8d98b79e596d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.582470 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.582454 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa7cb471-c256-411f-bf96-8d98b79e596d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s4bd8\" (UID: \"fa7cb471-c256-411f-bf96-8d98b79e596d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.599629 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.599605 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv67v\" (UniqueName: \"kubernetes.io/projected/fa7cb471-c256-411f-bf96-8d98b79e596d-kube-api-access-dv67v\") pod \"cert-manager-operator-controller-manager-55c444bf86-s4bd8\" (UID: \"fa7cb471-c256-411f-bf96-8d98b79e596d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.702953 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.702873 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" Jul 30 17:36:08.835331 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:08.835301 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8"] Jul 30 17:36:08.841334 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:36:08.841306 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7cb471_c256_411f_bf96_8d98b79e596d.slice/crio-fa842f0642efb9365f535316ad37b079459e777446cb3ad96c75f3057e1d17d5 WatchSource:0}: Error finding container fa842f0642efb9365f535316ad37b079459e777446cb3ad96c75f3057e1d17d5: Status 404 returned error can't find the container with id fa842f0642efb9365f535316ad37b079459e777446cb3ad96c75f3057e1d17d5 Jul 30 17:36:09.675103 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:09.675071 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" event={"ID":"fa7cb471-c256-411f-bf96-8d98b79e596d","Type":"ContainerStarted","Data":"fa842f0642efb9365f535316ad37b079459e777446cb3ad96c75f3057e1d17d5"} Jul 30 17:36:11.685827 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:11.685772 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" event={"ID":"fa7cb471-c256-411f-bf96-8d98b79e596d","Type":"ContainerStarted","Data":"97a09100a75ae723421a33250091b51b94fb0e6a55a8d4944d11d1a0e6e79134"} Jul 30 17:36:11.707597 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:11.707543 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s4bd8" podStartSLOduration=1.91490322 podStartE2EDuration="3.707529837s" totalImagesPullingTime="1.792626617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:11.705156796 +0000 UTC m=+566.437671481" watchObservedRunningTime="2026-07-30 17:36:11.707529837 +0000 UTC m=+566.440044520" Jul 30 17:36:12.672622 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.672586 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b"] Jul 30 17:36:12.674772 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.674757 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.677408 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.677387 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 17:36:12.677522 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.677496 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bj8qt\"" Jul 30 17:36:12.677570 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.677498 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 17:36:12.685081 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.685054 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b"] Jul 30 17:36:12.816707 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.816674 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.817118 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.816744 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w699p\" (UniqueName: \"kubernetes.io/projected/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-kube-api-access-w699p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.817118 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.816765 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.917468 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.917435 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w699p\" (UniqueName: \"kubernetes.io/projected/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-kube-api-access-w699p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.917468 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.917467 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.917641 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.917494 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.917855 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.917839 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.917921 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.917904 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.926594 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.926536 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w699p\" (UniqueName: \"kubernetes.io/projected/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-kube-api-access-w699p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:12.984796 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:12.984771 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:13.117229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:13.117146 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b"] Jul 30 17:36:13.119785 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:36:13.119759 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92126ec8_e45f_45b8_bea8_0f9e6ad0c0e0.slice/crio-fb7c09547c14670da41f551edc0cd2b6f1dfc614096aef09eb75b2b87782add8 WatchSource:0}: Error finding container fb7c09547c14670da41f551edc0cd2b6f1dfc614096aef09eb75b2b87782add8: Status 404 returned error can't find the container with id fb7c09547c14670da41f551edc0cd2b6f1dfc614096aef09eb75b2b87782add8 Jul 30 17:36:13.699597 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:13.699562 2550 generic.go:356] "Generic (PLEG): container finished" podID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerID="9a718135cc6ba4cb57672b3feeb1416f6a267d6baaee01993e77483afa8fc624" exitCode=0 Jul 30 17:36:13.699758 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:13.699621 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" event={"ID":"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0","Type":"ContainerDied","Data":"9a718135cc6ba4cb57672b3feeb1416f6a267d6baaee01993e77483afa8fc624"} Jul 30 17:36:13.699758 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:13.699682 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" event={"ID":"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0","Type":"ContainerStarted","Data":"fb7c09547c14670da41f551edc0cd2b6f1dfc614096aef09eb75b2b87782add8"} Jul 30 17:36:13.996333 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:13.996266 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rln9g"] Jul 30 17:36:13.998474 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:13.998455 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.001310 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.001273 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 30 17:36:14.001310 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.001291 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-j7q6v\"" Jul 30 17:36:14.001310 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.001301 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 30 17:36:14.009542 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.009524 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rln9g"] Jul 30 17:36:14.126348 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.126316 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzdbt\" (UniqueName: \"kubernetes.io/projected/81debd52-e228-40c9-8d1a-aa1aec992ce1-kube-api-access-fzdbt\") pod \"cert-manager-cainjector-559cdd8c87-rln9g\" (UID: \"81debd52-e228-40c9-8d1a-aa1aec992ce1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.126518 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.126391 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81debd52-e228-40c9-8d1a-aa1aec992ce1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rln9g\" (UID: \"81debd52-e228-40c9-8d1a-aa1aec992ce1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.227099 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.227056 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81debd52-e228-40c9-8d1a-aa1aec992ce1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rln9g\" (UID: \"81debd52-e228-40c9-8d1a-aa1aec992ce1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.227099 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.227106 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzdbt\" (UniqueName: \"kubernetes.io/projected/81debd52-e228-40c9-8d1a-aa1aec992ce1-kube-api-access-fzdbt\") pod \"cert-manager-cainjector-559cdd8c87-rln9g\" (UID: \"81debd52-e228-40c9-8d1a-aa1aec992ce1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.235524 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.235474 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81debd52-e228-40c9-8d1a-aa1aec992ce1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rln9g\" (UID: \"81debd52-e228-40c9-8d1a-aa1aec992ce1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.235637 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.235614 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzdbt\" (UniqueName: \"kubernetes.io/projected/81debd52-e228-40c9-8d1a-aa1aec992ce1-kube-api-access-fzdbt\") pod \"cert-manager-cainjector-559cdd8c87-rln9g\" (UID: \"81debd52-e228-40c9-8d1a-aa1aec992ce1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.314075 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.314016 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" Jul 30 17:36:14.462197 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.462156 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rln9g"] Jul 30 17:36:14.464856 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:36:14.464827 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81debd52_e228_40c9_8d1a_aa1aec992ce1.slice/crio-af1605f06df9d2c5f834728c17cbaad1b77e4349d0dbbc04266f96340e562414 WatchSource:0}: Error finding container af1605f06df9d2c5f834728c17cbaad1b77e4349d0dbbc04266f96340e562414: Status 404 returned error can't find the container with id af1605f06df9d2c5f834728c17cbaad1b77e4349d0dbbc04266f96340e562414 Jul 30 17:36:14.706631 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:14.706595 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" event={"ID":"81debd52-e228-40c9-8d1a-aa1aec992ce1","Type":"ContainerStarted","Data":"af1605f06df9d2c5f834728c17cbaad1b77e4349d0dbbc04266f96340e562414"} Jul 30 17:36:15.714448 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:15.714411 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" event={"ID":"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0","Type":"ContainerStarted","Data":"5ca94e71c0fd59c1813285f71fcedb1f919d048d2e72aeb96e34633702f551e6"} Jul 30 17:36:16.721404 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:16.721364 2550 generic.go:356] "Generic (PLEG): container finished" podID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerID="5ca94e71c0fd59c1813285f71fcedb1f919d048d2e72aeb96e34633702f551e6" exitCode=0 Jul 30 17:36:16.721884 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:16.721464 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" event={"ID":"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0","Type":"ContainerDied","Data":"5ca94e71c0fd59c1813285f71fcedb1f919d048d2e72aeb96e34633702f551e6"} Jul 30 17:36:17.729129 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:17.729076 2550 generic.go:356] "Generic (PLEG): container finished" podID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerID="f30b092a7c9845ac1e6034e7f7f038e67261e2eb77ce0c63c58a1e34af1593f7" exitCode=0 Jul 30 17:36:17.729543 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:17.729159 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" event={"ID":"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0","Type":"ContainerDied","Data":"f30b092a7c9845ac1e6034e7f7f038e67261e2eb77ce0c63c58a1e34af1593f7"} Jul 30 17:36:18.735306 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.735267 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" event={"ID":"81debd52-e228-40c9-8d1a-aa1aec992ce1","Type":"ContainerStarted","Data":"a9a9231ea80392385a453d13a464e163408cc21224e323a477cf5853ed610a24"} Jul 30 17:36:18.752744 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.752697 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rln9g" podStartSLOduration=2.260920119 podStartE2EDuration="5.752681099s" totalImagesPullingTime="3.49176098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:18.751031987 +0000 UTC m=+573.483546684" watchObservedRunningTime="2026-07-30 17:36:18.752681099 +0000 UTC m=+573.485195780" Jul 30 17:36:18.861935 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.861912 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:18.968678 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.968649 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-bundle\") pod \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " Jul 30 17:36:18.968863 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.968708 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-util\") pod \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " Jul 30 17:36:18.968863 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.968740 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w699p\" (UniqueName: \"kubernetes.io/projected/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-kube-api-access-w699p\") pod \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\" (UID: \"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0\") " Jul 30 17:36:18.969082 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.969046 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-bundle" (OuterVolumeSpecName: "bundle") pod "92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" (UID: "92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:36:18.970972 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.970949 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-kube-api-access-w699p" (OuterVolumeSpecName: "kube-api-access-w699p") pod "92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" (UID: "92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0"). InnerVolumeSpecName "kube-api-access-w699p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:36:18.973471 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:18.973441 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-util" (OuterVolumeSpecName: "util") pod "92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" (UID: "92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:36:19.069870 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.069779 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:19.069870 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.069801 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w699p\" (UniqueName: \"kubernetes.io/projected/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-kube-api-access-w699p\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:19.069870 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.069833 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:19.135800 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.135770 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fgkrl"] Jul 30 17:36:19.136108 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136094 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="util" Jul 30 17:36:19.136159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136110 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="util" Jul 30 17:36:19.136159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136128 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="pull" Jul 30 17:36:19.136159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136134 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="pull" Jul 30 17:36:19.136159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136142 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="extract" Jul 30 17:36:19.136159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136150 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="extract" Jul 30 17:36:19.136299 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.136190 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0" containerName="extract" Jul 30 17:36:19.140178 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.140160 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.142801 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.142783 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-89q8f\"" Jul 30 17:36:19.148020 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.147999 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fgkrl"] Jul 30 17:36:19.271221 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.271197 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9962b9d-0683-4917-b5ea-7077691c10c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fgkrl\" (UID: \"d9962b9d-0683-4917-b5ea-7077691c10c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.271340 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.271226 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5dkn\" (UniqueName: \"kubernetes.io/projected/d9962b9d-0683-4917-b5ea-7077691c10c7-kube-api-access-h5dkn\") pod \"cert-manager-webhook-6486ffd997-fgkrl\" (UID: \"d9962b9d-0683-4917-b5ea-7077691c10c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.372485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.372463 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9962b9d-0683-4917-b5ea-7077691c10c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fgkrl\" (UID: \"d9962b9d-0683-4917-b5ea-7077691c10c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.372585 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.372490 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5dkn\" (UniqueName: \"kubernetes.io/projected/d9962b9d-0683-4917-b5ea-7077691c10c7-kube-api-access-h5dkn\") pod \"cert-manager-webhook-6486ffd997-fgkrl\" (UID: \"d9962b9d-0683-4917-b5ea-7077691c10c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.381191 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.381165 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9962b9d-0683-4917-b5ea-7077691c10c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fgkrl\" (UID: \"d9962b9d-0683-4917-b5ea-7077691c10c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.381438 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.381417 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5dkn\" (UniqueName: \"kubernetes.io/projected/d9962b9d-0683-4917-b5ea-7077691c10c7-kube-api-access-h5dkn\") pod \"cert-manager-webhook-6486ffd997-fgkrl\" (UID: \"d9962b9d-0683-4917-b5ea-7077691c10c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.449242 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.449215 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.583349 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.583324 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fgkrl"] Jul 30 17:36:19.585636 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:36:19.585610 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9962b9d_0683_4917_b5ea_7077691c10c7.slice/crio-7ed54fe6a1501de4e980d1d361d9bcda21b0c204e6df604d7b6dc277f80c20c8 WatchSource:0}: Error finding container 7ed54fe6a1501de4e980d1d361d9bcda21b0c204e6df604d7b6dc277f80c20c8: Status 404 returned error can't find the container with id 7ed54fe6a1501de4e980d1d361d9bcda21b0c204e6df604d7b6dc277f80c20c8 Jul 30 17:36:19.742177 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.742148 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" event={"ID":"d9962b9d-0683-4917-b5ea-7077691c10c7","Type":"ContainerStarted","Data":"40b06e20f162b090a710de7b0a8368f30c92ac54603ddf69d81b3ec5c9f3685b"} Jul 30 17:36:19.742604 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.742186 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" event={"ID":"d9962b9d-0683-4917-b5ea-7077691c10c7","Type":"ContainerStarted","Data":"7ed54fe6a1501de4e980d1d361d9bcda21b0c204e6df604d7b6dc277f80c20c8"} Jul 30 17:36:19.742604 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.742221 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:19.744214 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.744193 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" Jul 30 17:36:19.744328 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.744213 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2xc8b" event={"ID":"92126ec8-e45f-45b8-bea8-0f9e6ad0c0e0","Type":"ContainerDied","Data":"fb7c09547c14670da41f551edc0cd2b6f1dfc614096aef09eb75b2b87782add8"} Jul 30 17:36:19.744328 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.744249 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb7c09547c14670da41f551edc0cd2b6f1dfc614096aef09eb75b2b87782add8" Jul 30 17:36:19.759888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:19.759840 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" podStartSLOduration=0.759805275 podStartE2EDuration="759.805275ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:19.759263406 +0000 UTC m=+574.491778090" watchObservedRunningTime="2026-07-30 17:36:19.759805275 +0000 UTC m=+574.492319957" Jul 30 17:36:25.750257 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:25.750229 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fgkrl" Jul 30 17:36:38.222166 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.222128 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts"] Jul 30 17:36:38.226660 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.226641 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.230116 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.230094 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 17:36:38.231265 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.231249 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bj8qt\"" Jul 30 17:36:38.231327 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.231249 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 17:36:38.235719 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.235698 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts"] Jul 30 17:36:38.314552 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.314522 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.314552 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.314553 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.314713 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.314584 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54jkv\" (UniqueName: \"kubernetes.io/projected/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-kube-api-access-54jkv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.415566 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.415530 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.415739 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.415587 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.415739 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.415643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54jkv\" (UniqueName: \"kubernetes.io/projected/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-kube-api-access-54jkv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.416025 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.416005 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.416068 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.416012 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.424970 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.424940 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54jkv\" (UniqueName: \"kubernetes.io/projected/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-kube-api-access-54jkv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.536252 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.536164 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:38.670641 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.670616 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts"] Jul 30 17:36:38.672915 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:36:38.672888 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41cc1f97_e9ff_42a2_9a88_a7db10e8e5d8.slice/crio-33597fc7b7e8f042fb67cdaea65821e64687dad1bc0909962efde5038c063b1a WatchSource:0}: Error finding container 33597fc7b7e8f042fb67cdaea65821e64687dad1bc0909962efde5038c063b1a: Status 404 returned error can't find the container with id 33597fc7b7e8f042fb67cdaea65821e64687dad1bc0909962efde5038c063b1a Jul 30 17:36:38.824934 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.824870 2550 generic.go:356] "Generic (PLEG): container finished" podID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerID="91531664bd8fd4823960ff96c4edf71f58ddbf6bd43199b03084d65dfb5656a4" exitCode=0 Jul 30 17:36:38.825037 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.824946 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" event={"ID":"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8","Type":"ContainerDied","Data":"91531664bd8fd4823960ff96c4edf71f58ddbf6bd43199b03084d65dfb5656a4"} Jul 30 17:36:38.825037 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:38.825017 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" event={"ID":"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8","Type":"ContainerStarted","Data":"33597fc7b7e8f042fb67cdaea65821e64687dad1bc0909962efde5038c063b1a"} Jul 30 17:36:39.830971 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:39.830877 2550 generic.go:356] "Generic (PLEG): container finished" podID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerID="5e31146a46b087b741da08c21ec626cea7e91d79688b75a2062c2eb62462a82c" exitCode=0 Jul 30 17:36:39.830971 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:39.830953 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" event={"ID":"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8","Type":"ContainerDied","Data":"5e31146a46b087b741da08c21ec626cea7e91d79688b75a2062c2eb62462a82c"} Jul 30 17:36:40.837774 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:40.837741 2550 generic.go:356] "Generic (PLEG): container finished" podID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerID="cd0e5813695a2decbf114e8e4acc81aa1ff462dd2e3810e087f99648b158f9ed" exitCode=0 Jul 30 17:36:40.838155 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:40.837794 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" event={"ID":"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8","Type":"ContainerDied","Data":"cd0e5813695a2decbf114e8e4acc81aa1ff462dd2e3810e087f99648b158f9ed"} Jul 30 17:36:41.965137 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:41.965112 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:42.040667 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.040637 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-util\") pod \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " Jul 30 17:36:42.040806 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.040713 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-bundle\") pod \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " Jul 30 17:36:42.040806 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.040746 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54jkv\" (UniqueName: \"kubernetes.io/projected/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-kube-api-access-54jkv\") pod \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\" (UID: \"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8\") " Jul 30 17:36:42.041396 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.041365 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-bundle" (OuterVolumeSpecName: "bundle") pod "41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" (UID: "41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:36:42.042881 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.042861 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-kube-api-access-54jkv" (OuterVolumeSpecName: "kube-api-access-54jkv") pod "41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" (UID: "41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8"). InnerVolumeSpecName "kube-api-access-54jkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:36:42.048499 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.048477 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-util" (OuterVolumeSpecName: "util") pod "41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" (UID: "41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:36:42.141403 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.141330 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:42.141403 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.141355 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:42.141403 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.141367 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54jkv\" (UniqueName: \"kubernetes.io/projected/41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8-kube-api-access-54jkv\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:36:42.849870 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.849841 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" Jul 30 17:36:42.850012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.849840 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afp94ts" event={"ID":"41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8","Type":"ContainerDied","Data":"33597fc7b7e8f042fb67cdaea65821e64687dad1bc0909962efde5038c063b1a"} Jul 30 17:36:42.850012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:42.849957 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33597fc7b7e8f042fb67cdaea65821e64687dad1bc0909962efde5038c063b1a" Jul 30 17:36:49.600607 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600569 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b"] Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600878 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="util" Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600890 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="util" Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600900 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="extract" Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600904 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="extract" Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600925 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="pull" Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600930 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="pull" Jul 30 17:36:49.601862 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.600977 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="41cc1f97-e9ff-42a2-9a88-a7db10e8e5d8" containerName="extract" Jul 30 17:36:49.602717 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.602701 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.606416 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.606394 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 17:36:49.606599 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.606587 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-thdx9\"" Jul 30 17:36:49.606737 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.606722 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 17:36:49.606830 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.606794 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 17:36:49.609343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.609310 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 17:36:49.618587 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.618564 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b"] Jul 30 17:36:49.702332 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.702304 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrh5\" (UniqueName: \"kubernetes.io/projected/aceb3ec2-9767-473a-bf13-085f5c70815d-kube-api-access-nqrh5\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.702456 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.702357 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aceb3ec2-9767-473a-bf13-085f5c70815d-tmp\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.702456 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.702381 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aceb3ec2-9767-473a-bf13-085f5c70815d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.702456 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.702403 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aceb3ec2-9767-473a-bf13-085f5c70815d-webhook-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.702456 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.702439 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aceb3ec2-9767-473a-bf13-085f5c70815d-manifests\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.802748 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.802721 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aceb3ec2-9767-473a-bf13-085f5c70815d-tmp\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.802898 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.802750 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aceb3ec2-9767-473a-bf13-085f5c70815d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.802898 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.802775 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aceb3ec2-9767-473a-bf13-085f5c70815d-webhook-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.802898 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.802843 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aceb3ec2-9767-473a-bf13-085f5c70815d-manifests\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.802898 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.802869 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrh5\" (UniqueName: \"kubernetes.io/projected/aceb3ec2-9767-473a-bf13-085f5c70815d-kube-api-access-nqrh5\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.803239 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.803208 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aceb3ec2-9767-473a-bf13-085f5c70815d-manifests\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.805135 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.805102 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aceb3ec2-9767-473a-bf13-085f5c70815d-tmp\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.805255 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.805237 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aceb3ec2-9767-473a-bf13-085f5c70815d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.805376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.805355 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aceb3ec2-9767-473a-bf13-085f5c70815d-webhook-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.811159 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.811140 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrh5\" (UniqueName: \"kubernetes.io/projected/aceb3ec2-9767-473a-bf13-085f5c70815d-kube-api-access-nqrh5\") pod \"opendatahub-operator-controller-manager-975dfc96f-srl2b\" (UID: \"aceb3ec2-9767-473a-bf13-085f5c70815d\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:49.912505 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:49.912445 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:50.049231 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:50.049201 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b"] Jul 30 17:36:50.052906 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:36:50.052875 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaceb3ec2_9767_473a_bf13_085f5c70815d.slice/crio-6dc7401cef9e4f3caded8ff2888854aa76fe15a221980681d3c217fe778b9bee WatchSource:0}: Error finding container 6dc7401cef9e4f3caded8ff2888854aa76fe15a221980681d3c217fe778b9bee: Status 404 returned error can't find the container with id 6dc7401cef9e4f3caded8ff2888854aa76fe15a221980681d3c217fe778b9bee Jul 30 17:36:50.055126 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:50.055107 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 17:36:50.891242 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:50.891200 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" event={"ID":"aceb3ec2-9767-473a-bf13-085f5c70815d","Type":"ContainerStarted","Data":"6dc7401cef9e4f3caded8ff2888854aa76fe15a221980681d3c217fe778b9bee"} Jul 30 17:36:53.906974 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:53.906937 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" event={"ID":"aceb3ec2-9767-473a-bf13-085f5c70815d","Type":"ContainerStarted","Data":"c39404d724944c4c4e099676d55a610914f96da626a6c839fff12d1cb11e85b0"} Jul 30 17:36:54.913092 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:54.913053 2550 generic.go:356] "Generic (PLEG): container finished" podID="aceb3ec2-9767-473a-bf13-085f5c70815d" containerID="c39404d724944c4c4e099676d55a610914f96da626a6c839fff12d1cb11e85b0" exitCode=0 Jul 30 17:36:54.913494 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:54.913098 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" event={"ID":"aceb3ec2-9767-473a-bf13-085f5c70815d","Type":"ContainerDied","Data":"c39404d724944c4c4e099676d55a610914f96da626a6c839fff12d1cb11e85b0"} Jul 30 17:36:55.918929 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:55.918901 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" event={"ID":"aceb3ec2-9767-473a-bf13-085f5c70815d","Type":"ContainerStarted","Data":"666aa5935de79afebe43e31e846e8f6d52675ca0c81162eaa1bc5383b6d68f81"} Jul 30 17:36:55.919302 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:55.918966 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:36:55.943748 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:36:55.943704 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" podStartSLOduration=2.501548874 podStartE2EDuration="6.943688994s" totalImagesPullingTime="3.378558152s" totalInitContainerRuntime="1.063581968s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:55.942261434 +0000 UTC m=+610.674776119" watchObservedRunningTime="2026-07-30 17:36:55.943688994 +0000 UTC m=+610.676203677" Jul 30 17:37:06.925687 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:06.925660 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-srl2b" Jul 30 17:37:09.501716 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.501684 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr"] Jul 30 17:37:09.504853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.504836 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.507756 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.507736 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 30 17:37:09.509109 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.509088 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 30 17:37:09.509208 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.509099 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 30 17:37:09.509274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.509124 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 30 17:37:09.509274 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.509132 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rk2zb\"" Jul 30 17:37:09.509376 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.509167 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 30 17:37:09.518311 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.518291 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr"] Jul 30 17:37:09.663486 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.663451 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/60ad5ce0-3621-48a5-a324-65bb968b8589-metrics-cert\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.663644 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.663498 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/60ad5ce0-3621-48a5-a324-65bb968b8589-manager-config\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.663644 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.663589 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60ad5ce0-3621-48a5-a324-65bb968b8589-cert\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.663644 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.663628 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkqn4\" (UniqueName: \"kubernetes.io/projected/60ad5ce0-3621-48a5-a324-65bb968b8589-kube-api-access-zkqn4\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.764404 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.764327 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60ad5ce0-3621-48a5-a324-65bb968b8589-cert\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.764404 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.764365 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkqn4\" (UniqueName: \"kubernetes.io/projected/60ad5ce0-3621-48a5-a324-65bb968b8589-kube-api-access-zkqn4\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.764615 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.764408 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/60ad5ce0-3621-48a5-a324-65bb968b8589-metrics-cert\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.764615 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.764429 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/60ad5ce0-3621-48a5-a324-65bb968b8589-manager-config\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.765184 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.765162 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/60ad5ce0-3621-48a5-a324-65bb968b8589-manager-config\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.767467 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.767446 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60ad5ce0-3621-48a5-a324-65bb968b8589-cert\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.767572 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.767509 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/60ad5ce0-3621-48a5-a324-65bb968b8589-metrics-cert\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.780164 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.780142 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkqn4\" (UniqueName: \"kubernetes.io/projected/60ad5ce0-3621-48a5-a324-65bb968b8589-kube-api-access-zkqn4\") pod \"lws-controller-manager-5cdfbb974d-mvjbr\" (UID: \"60ad5ce0-3621-48a5-a324-65bb968b8589\") " pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.813965 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.813941 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:09.959342 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.959318 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr"] Jul 30 17:37:09.961460 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:37:09.961428 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ad5ce0_3621_48a5_a324_65bb968b8589.slice/crio-3b541b7b9b2b7ef444f8bde350482ec87caef413151f9757a019b0f283819efc WatchSource:0}: Error finding container 3b541b7b9b2b7ef444f8bde350482ec87caef413151f9757a019b0f283819efc: Status 404 returned error can't find the container with id 3b541b7b9b2b7ef444f8bde350482ec87caef413151f9757a019b0f283819efc Jul 30 17:37:09.978655 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:09.978623 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" event={"ID":"60ad5ce0-3621-48a5-a324-65bb968b8589","Type":"ContainerStarted","Data":"3b541b7b9b2b7ef444f8bde350482ec87caef413151f9757a019b0f283819efc"} Jul 30 17:37:11.992979 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:11.992941 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" event={"ID":"60ad5ce0-3621-48a5-a324-65bb968b8589","Type":"ContainerStarted","Data":"0c6a9383a367ff616216dbccc28b66195d768388e5a6b6fbbeaa66bfaf82fd74"} Jul 30 17:37:11.993453 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:11.993008 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:12.010758 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:12.010710 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" podStartSLOduration=1.5024071220000002 podStartE2EDuration="3.010693944s" totalImagesPullingTime="1.508286822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:12.008944635 +0000 UTC m=+626.741459353" watchObservedRunningTime="2026-07-30 17:37:12.010693944 +0000 UTC m=+626.743208627" Jul 30 17:37:16.958747 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:16.958713 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg"] Jul 30 17:37:16.962513 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:16.962494 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:16.965467 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:16.965442 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 17:37:16.965569 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:16.965557 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-m2hq6\"" Jul 30 17:37:16.979776 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:16.979752 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg"] Jul 30 17:37:17.034267 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034233 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034426 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034281 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034426 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034310 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034426 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034336 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034426 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034360 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034426 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034401 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034650 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034441 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034650 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034458 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rf8f\" (UniqueName: \"kubernetes.io/projected/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-kube-api-access-9rf8f\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.034650 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.034534 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135365 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135322 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135560 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135403 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135560 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135427 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rf8f\" (UniqueName: \"kubernetes.io/projected/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-kube-api-access-9rf8f\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135560 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135473 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135696 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135671 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135746 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135729 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135801 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135760 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135877 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135793 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135877 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135800 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.135877 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.135846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.136044 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.136023 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.136110 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.136041 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.136110 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.136096 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.136463 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.136444 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.138288 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.138266 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.138374 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.138306 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.153395 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.153368 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rf8f\" (UniqueName: \"kubernetes.io/projected/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-kube-api-access-9rf8f\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.159255 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.159224 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg\" (UID: \"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.273456 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.273360 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:17.415185 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:17.415163 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg"] Jul 30 17:37:17.417240 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:37:17.417216 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dab0ff8_c7ef_4ad3_bf11_91e10f6774d9.slice/crio-0dbabc454d59ee7f3ec81515ac3318d33a278bbde954d2ac3d3a7c6728e0cbd0 WatchSource:0}: Error finding container 0dbabc454d59ee7f3ec81515ac3318d33a278bbde954d2ac3d3a7c6728e0cbd0: Status 404 returned error can't find the container with id 0dbabc454d59ee7f3ec81515ac3318d33a278bbde954d2ac3d3a7c6728e0cbd0 Jul 30 17:37:18.023718 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:18.023685 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" event={"ID":"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9","Type":"ContainerStarted","Data":"0dbabc454d59ee7f3ec81515ac3318d33a278bbde954d2ac3d3a7c6728e0cbd0"} Jul 30 17:37:19.642836 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.642197 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9"] Jul 30 17:37:19.650435 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.650397 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.653153 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.653123 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 17:37:19.653287 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.653235 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 17:37:19.653287 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.653235 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 17:37:19.653657 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.653637 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qxt7c\"" Jul 30 17:37:19.655746 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655713 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dfaea14-d720-462a-9504-602c26734240-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.655879 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655753 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dfaea14-d720-462a-9504-602c26734240-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.655879 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655777 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.655879 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655805 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.655879 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655853 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4dfaea14-d720-462a-9504-602c26734240-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.656092 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655878 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kh5z\" (UniqueName: \"kubernetes.io/projected/4dfaea14-d720-462a-9504-602c26734240-kube-api-access-8kh5z\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.656092 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.655957 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.659879 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.659347 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9"] Jul 30 17:37:19.756303 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756271 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.756519 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756339 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dfaea14-d720-462a-9504-602c26734240-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.756519 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dfaea14-d720-462a-9504-602c26734240-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.756519 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756397 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.756519 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756422 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.756737 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756583 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4dfaea14-d720-462a-9504-602c26734240-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.756737 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.756627 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kh5z\" (UniqueName: \"kubernetes.io/projected/4dfaea14-d720-462a-9504-602c26734240-kube-api-access-8kh5z\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.757643 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.757588 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4dfaea14-d720-462a-9504-602c26734240-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.759178 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.759129 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dfaea14-d720-462a-9504-602c26734240-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.759301 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.759281 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.759373 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.759326 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.759488 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.759466 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4dfaea14-d720-462a-9504-602c26734240-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.772893 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.772871 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dfaea14-d720-462a-9504-602c26734240-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.773156 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.773132 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kh5z\" (UniqueName: \"kubernetes.io/projected/4dfaea14-d720-462a-9504-602c26734240-kube-api-access-8kh5z\") pod \"istiod-openshift-gateway-5677ccb49b-8qgj9\" (UID: \"4dfaea14-d720-462a-9504-602c26734240\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:19.964012 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:19.963935 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:20.303720 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:20.303687 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:37:20.303843 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:20.303754 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:37:20.303843 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:20.303785 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:37:20.371674 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:20.371652 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9"] Jul 30 17:37:20.373419 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:37:20.373398 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dfaea14_d720_462a_9504_602c26734240.slice/crio-81a4cd2e40ecf4c5343006e4d5eb9d2318063a186c4054b8fb41e9cfba7b3920 WatchSource:0}: Error finding container 81a4cd2e40ecf4c5343006e4d5eb9d2318063a186c4054b8fb41e9cfba7b3920: Status 404 returned error can't find the container with id 81a4cd2e40ecf4c5343006e4d5eb9d2318063a186c4054b8fb41e9cfba7b3920 Jul 30 17:37:21.041800 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:21.041761 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" event={"ID":"4dfaea14-d720-462a-9504-602c26734240","Type":"ContainerStarted","Data":"81a4cd2e40ecf4c5343006e4d5eb9d2318063a186c4054b8fb41e9cfba7b3920"} Jul 30 17:37:21.047849 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:21.046088 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" event={"ID":"3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9","Type":"ContainerStarted","Data":"b667cedb2c0791bd3fae66141d41a612b794bcdba2db586dc5cf6308856cf486"} Jul 30 17:37:21.107890 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:21.107834 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" podStartSLOduration=2.223405004 podStartE2EDuration="5.107793167s" totalImagesPullingTime="2.884388163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:21.106004815 +0000 UTC m=+635.838519501" watchObservedRunningTime="2026-07-30 17:37:21.107793167 +0000 UTC m=+635.840307854" Jul 30 17:37:21.273786 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:21.273707 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:21.282582 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:21.282529 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:22.052102 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:22.052070 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:22.053331 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:22.053305 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg" Jul 30 17:37:22.890520 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:22.890489 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:37:22.890633 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:22.890561 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:37:23.000449 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:23.000415 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5cdfbb974d-mvjbr" Jul 30 17:37:23.066788 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:23.066748 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" event={"ID":"4dfaea14-d720-462a-9504-602c26734240","Type":"ContainerStarted","Data":"190d19f1c02a9614c8dc787dfeafd8427627950163d1a8d498f2bcc3d74ed6e4"} Jul 30 17:37:23.067246 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:23.066915 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:37:23.068241 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:23.068216 2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-8qgj9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 17:37:23.068362 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:23.068266 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" podUID="4dfaea14-d720-462a-9504-602c26734240" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 17:37:23.091525 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:23.091471 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" podStartSLOduration=1.5766040399999999 podStartE2EDuration="4.091457791s" totalImagesPullingTime="2.514853751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:23.089397832 +0000 UTC m=+637.821912513" watchObservedRunningTime="2026-07-30 17:37:23.091457791 +0000 UTC m=+637.823972472" Jul 30 17:37:24.071495 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:37:24.071467 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8qgj9" Jul 30 17:38:26.220054 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.220021 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf"] Jul 30 17:38:26.222525 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.222509 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.225441 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.225419 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 30 17:38:26.225557 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.225419 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 30 17:38:26.225557 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.225503 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wcchl\"" Jul 30 17:38:26.232747 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.232722 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf"] Jul 30 17:38:26.319113 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.319084 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.319263 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.319117 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.319263 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.319221 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7j67\" (UniqueName: \"kubernetes.io/projected/4d495923-6d40-4598-b619-679fd72ff675-kube-api-access-t7j67\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.420240 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.420207 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.420391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.420246 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.420391 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.420293 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7j67\" (UniqueName: \"kubernetes.io/projected/4d495923-6d40-4598-b619-679fd72ff675-kube-api-access-t7j67\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.420607 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.420576 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.420658 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.420642 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.429427 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.429408 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7j67\" (UniqueName: \"kubernetes.io/projected/4d495923-6d40-4598-b619-679fd72ff675-kube-api-access-t7j67\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.531628 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.531549 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:26.605876 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.605844 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7"] Jul 30 17:38:26.608834 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.608793 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.618492 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.618474 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7"] Jul 30 17:38:26.670621 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.670602 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf"] Jul 30 17:38:26.672759 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:38:26.672731 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d495923_6d40_4598_b619_679fd72ff675.slice/crio-9a99ed2eaf35c4116c0709f06e1eca4d909de79652ee6a7a596bbf08c3e71b43 WatchSource:0}: Error finding container 9a99ed2eaf35c4116c0709f06e1eca4d909de79652ee6a7a596bbf08c3e71b43: Status 404 returned error can't find the container with id 9a99ed2eaf35c4116c0709f06e1eca4d909de79652ee6a7a596bbf08c3e71b43 Jul 30 17:38:26.723792 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.723766 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg7hq\" (UniqueName: \"kubernetes.io/projected/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-kube-api-access-hg7hq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.723920 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.723906 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.723973 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.723959 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.824912 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.824793 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7hq\" (UniqueName: \"kubernetes.io/projected/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-kube-api-access-hg7hq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.824912 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.824874 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.824912 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.824910 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.825261 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.825246 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.825317 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.825286 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.839071 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.839039 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7hq\" (UniqueName: \"kubernetes.io/projected/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-kube-api-access-hg7hq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:26.921019 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:26.920983 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:27.011030 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.010999 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m"] Jul 30 17:38:27.013771 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.013751 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.022067 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.022040 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m"] Jul 30 17:38:27.059095 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.059068 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7"] Jul 30 17:38:27.060848 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:38:27.060830 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87acff2_b3aa_4e46_a89b_4a48ae2f13d9.slice/crio-338279861f43eec931bc22127b627e67c46bf1f8e90c2676c4062e4bae0954d9 WatchSource:0}: Error finding container 338279861f43eec931bc22127b627e67c46bf1f8e90c2676c4062e4bae0954d9: Status 404 returned error can't find the container with id 338279861f43eec931bc22127b627e67c46bf1f8e90c2676c4062e4bae0954d9 Jul 30 17:38:27.127565 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.127531 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.127727 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.127585 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l678v\" (UniqueName: \"kubernetes.io/projected/efdb1c15-eb77-4a88-aa23-9125367e7991-kube-api-access-l678v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.127727 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.127663 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.229193 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.229155 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l678v\" (UniqueName: \"kubernetes.io/projected/efdb1c15-eb77-4a88-aa23-9125367e7991-kube-api-access-l678v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.229683 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.229212 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.229683 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.229294 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.229683 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.229635 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.229804 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.229684 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.238443 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.238419 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l678v\" (UniqueName: \"kubernetes.io/projected/efdb1c15-eb77-4a88-aa23-9125367e7991-kube-api-access-l678v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.325727 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.325691 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:27.356250 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.356216 2550 generic.go:356] "Generic (PLEG): container finished" podID="4d495923-6d40-4598-b619-679fd72ff675" containerID="1c611f6f630bed30268d95eea527b592ec5031fd6e58d4001cd964799cf00c91" exitCode=0 Jul 30 17:38:27.356416 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.356298 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" event={"ID":"4d495923-6d40-4598-b619-679fd72ff675","Type":"ContainerDied","Data":"1c611f6f630bed30268d95eea527b592ec5031fd6e58d4001cd964799cf00c91"} Jul 30 17:38:27.356416 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.356387 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" event={"ID":"4d495923-6d40-4598-b619-679fd72ff675","Type":"ContainerStarted","Data":"9a99ed2eaf35c4116c0709f06e1eca4d909de79652ee6a7a596bbf08c3e71b43"} Jul 30 17:38:27.358366 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.358341 2550 generic.go:356] "Generic (PLEG): container finished" podID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerID="f418eb64e77c3c14c3a5f0f751b21bb35e670573e16981fef2551cbd96e8e661" exitCode=0 Jul 30 17:38:27.358464 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.358395 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" event={"ID":"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9","Type":"ContainerDied","Data":"f418eb64e77c3c14c3a5f0f751b21bb35e670573e16981fef2551cbd96e8e661"} Jul 30 17:38:27.358464 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.358432 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" event={"ID":"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9","Type":"ContainerStarted","Data":"338279861f43eec931bc22127b627e67c46bf1f8e90c2676c4062e4bae0954d9"} Jul 30 17:38:27.472410 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.472374 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m"] Jul 30 17:38:27.476180 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:38:27.476154 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefdb1c15_eb77_4a88_aa23_9125367e7991.slice/crio-827bf1b061aaf40a725002c61f435f2e5f6506cf6dfd713184a307f504b51b39 WatchSource:0}: Error finding container 827bf1b061aaf40a725002c61f435f2e5f6506cf6dfd713184a307f504b51b39: Status 404 returned error can't find the container with id 827bf1b061aaf40a725002c61f435f2e5f6506cf6dfd713184a307f504b51b39 Jul 30 17:38:27.609895 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.609859 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf"] Jul 30 17:38:27.612398 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.612380 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.620911 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.620886 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf"] Jul 30 17:38:27.734402 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.734369 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.734571 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.734414 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.734571 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.734448 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj9gm\" (UniqueName: \"kubernetes.io/projected/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-kube-api-access-nj9gm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.835154 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.835083 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.835154 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.835126 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.835326 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.835161 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj9gm\" (UniqueName: \"kubernetes.io/projected/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-kube-api-access-nj9gm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.835438 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.835421 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.835511 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.835493 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.843934 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.843913 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj9gm\" (UniqueName: \"kubernetes.io/projected/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-kube-api-access-nj9gm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:27.946383 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:27.946348 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:28.143512 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.143450 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf"] Jul 30 17:38:28.155231 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:38:28.155200 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7b3845_a35e_4eac_ac2c_c9c81b2dbbdc.slice/crio-253350bb4c44c34b1664c057b8f75cefe7dc877d8d90962c9586e17f997e7b97 WatchSource:0}: Error finding container 253350bb4c44c34b1664c057b8f75cefe7dc877d8d90962c9586e17f997e7b97: Status 404 returned error can't find the container with id 253350bb4c44c34b1664c057b8f75cefe7dc877d8d90962c9586e17f997e7b97 Jul 30 17:38:28.364160 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.364130 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerID="f996246616103dfa1b5b98bf141b692af1f47a9725eae075a3b07a6edb27cdfe" exitCode=0 Jul 30 17:38:28.364596 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.364206 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" event={"ID":"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc","Type":"ContainerDied","Data":"f996246616103dfa1b5b98bf141b692af1f47a9725eae075a3b07a6edb27cdfe"} Jul 30 17:38:28.364596 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.364268 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" event={"ID":"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc","Type":"ContainerStarted","Data":"253350bb4c44c34b1664c057b8f75cefe7dc877d8d90962c9586e17f997e7b97"} Jul 30 17:38:28.366360 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.366322 2550 generic.go:356] "Generic (PLEG): container finished" podID="4d495923-6d40-4598-b619-679fd72ff675" containerID="c5787b3ce3b314a497404d6bd44b4146fa20e8bbee7237e10002b662cdc07a1a" exitCode=0 Jul 30 17:38:28.366498 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.366370 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" event={"ID":"4d495923-6d40-4598-b619-679fd72ff675","Type":"ContainerDied","Data":"c5787b3ce3b314a497404d6bd44b4146fa20e8bbee7237e10002b662cdc07a1a"} Jul 30 17:38:28.368229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.368087 2550 generic.go:356] "Generic (PLEG): container finished" podID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerID="2da39164b31d039a23a48b9eb55367541bbf500aa21b35a393f8c421b772d050" exitCode=0 Jul 30 17:38:28.368229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.368170 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" event={"ID":"efdb1c15-eb77-4a88-aa23-9125367e7991","Type":"ContainerDied","Data":"2da39164b31d039a23a48b9eb55367541bbf500aa21b35a393f8c421b772d050"} Jul 30 17:38:28.368229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:28.368203 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" event={"ID":"efdb1c15-eb77-4a88-aa23-9125367e7991","Type":"ContainerStarted","Data":"827bf1b061aaf40a725002c61f435f2e5f6506cf6dfd713184a307f504b51b39"} Jul 30 17:38:29.375036 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.374997 2550 generic.go:356] "Generic (PLEG): container finished" podID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerID="2e2473558ef9e9ed6a68b01710e10f4e2949ac98fbe9c73adefc815c2f7ecf82" exitCode=0 Jul 30 17:38:29.375465 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.375081 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" event={"ID":"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9","Type":"ContainerDied","Data":"2e2473558ef9e9ed6a68b01710e10f4e2949ac98fbe9c73adefc815c2f7ecf82"} Jul 30 17:38:29.377077 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.377049 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerID="568fcbb2ec92fbf42fb8aed97f1fb0c3e4fe5cdbbf2fb35222edbd97a3acb36c" exitCode=0 Jul 30 17:38:29.377188 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.377138 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" event={"ID":"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc","Type":"ContainerDied","Data":"568fcbb2ec92fbf42fb8aed97f1fb0c3e4fe5cdbbf2fb35222edbd97a3acb36c"} Jul 30 17:38:29.379346 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.379327 2550 generic.go:356] "Generic (PLEG): container finished" podID="4d495923-6d40-4598-b619-679fd72ff675" containerID="e883da847138edc07f218f4fce21d5f795c137c5c1ead69f7fcc5fa9e7181448" exitCode=0 Jul 30 17:38:29.379448 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.379390 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" event={"ID":"4d495923-6d40-4598-b619-679fd72ff675","Type":"ContainerDied","Data":"e883da847138edc07f218f4fce21d5f795c137c5c1ead69f7fcc5fa9e7181448"} Jul 30 17:38:29.381141 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.381124 2550 generic.go:356] "Generic (PLEG): container finished" podID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerID="ab8fe569d008eb0043f4ee96622ac9220bc1211a2e881ab679a2346917188668" exitCode=0 Jul 30 17:38:29.381236 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:29.381183 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" event={"ID":"efdb1c15-eb77-4a88-aa23-9125367e7991","Type":"ContainerDied","Data":"ab8fe569d008eb0043f4ee96622ac9220bc1211a2e881ab679a2346917188668"} Jul 30 17:38:30.391120 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.391084 2550 generic.go:356] "Generic (PLEG): container finished" podID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerID="a8952d9a3e8da056ccc3f62d039daffed02f491a104b6701a759fc75d040cba0" exitCode=0 Jul 30 17:38:30.391554 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.391168 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" event={"ID":"efdb1c15-eb77-4a88-aa23-9125367e7991","Type":"ContainerDied","Data":"a8952d9a3e8da056ccc3f62d039daffed02f491a104b6701a759fc75d040cba0"} Jul 30 17:38:30.393234 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.393207 2550 generic.go:356] "Generic (PLEG): container finished" podID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerID="e092a552391a8b2049219cd3bb1a68d69761191d42a79bab8a9ba10ee70b489e" exitCode=0 Jul 30 17:38:30.393337 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.393299 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" event={"ID":"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9","Type":"ContainerDied","Data":"e092a552391a8b2049219cd3bb1a68d69761191d42a79bab8a9ba10ee70b489e"} Jul 30 17:38:30.395211 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.395193 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerID="0b7be19443f75a1a4d5e8f7aa7526a9ba034d77fa425126fa22f517875cb4326" exitCode=0 Jul 30 17:38:30.395302 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.395254 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" event={"ID":"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc","Type":"ContainerDied","Data":"0b7be19443f75a1a4d5e8f7aa7526a9ba034d77fa425126fa22f517875cb4326"} Jul 30 17:38:30.527059 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.527032 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:30.561390 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.561363 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-bundle\") pod \"4d495923-6d40-4598-b619-679fd72ff675\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " Jul 30 17:38:30.561542 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.561399 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-util\") pod \"4d495923-6d40-4598-b619-679fd72ff675\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " Jul 30 17:38:30.561542 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.561464 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7j67\" (UniqueName: \"kubernetes.io/projected/4d495923-6d40-4598-b619-679fd72ff675-kube-api-access-t7j67\") pod \"4d495923-6d40-4598-b619-679fd72ff675\" (UID: \"4d495923-6d40-4598-b619-679fd72ff675\") " Jul 30 17:38:30.562006 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.561974 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-bundle" (OuterVolumeSpecName: "bundle") pod "4d495923-6d40-4598-b619-679fd72ff675" (UID: "4d495923-6d40-4598-b619-679fd72ff675"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:30.564005 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.563983 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d495923-6d40-4598-b619-679fd72ff675-kube-api-access-t7j67" (OuterVolumeSpecName: "kube-api-access-t7j67") pod "4d495923-6d40-4598-b619-679fd72ff675" (UID: "4d495923-6d40-4598-b619-679fd72ff675"). InnerVolumeSpecName "kube-api-access-t7j67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:38:30.570657 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.570629 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-util" (OuterVolumeSpecName: "util") pod "4d495923-6d40-4598-b619-679fd72ff675" (UID: "4d495923-6d40-4598-b619-679fd72ff675"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:30.662859 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.662764 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:30.662859 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.662789 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d495923-6d40-4598-b619-679fd72ff675-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:30.662859 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:30.662799 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7j67\" (UniqueName: \"kubernetes.io/projected/4d495923-6d40-4598-b619-679fd72ff675-kube-api-access-t7j67\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.402448 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.402415 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" Jul 30 17:38:31.402448 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.402428 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf" event={"ID":"4d495923-6d40-4598-b619-679fd72ff675","Type":"ContainerDied","Data":"9a99ed2eaf35c4116c0709f06e1eca4d909de79652ee6a7a596bbf08c3e71b43"} Jul 30 17:38:31.402896 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.402475 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a99ed2eaf35c4116c0709f06e1eca4d909de79652ee6a7a596bbf08c3e71b43" Jul 30 17:38:31.555629 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.555604 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:31.585215 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.585189 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:31.588445 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.588425 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:31.671644 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671572 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-util\") pod \"efdb1c15-eb77-4a88-aa23-9125367e7991\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " Jul 30 17:38:31.671644 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671624 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg7hq\" (UniqueName: \"kubernetes.io/projected/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-kube-api-access-hg7hq\") pod \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " Jul 30 17:38:31.671866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671648 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-bundle\") pod \"efdb1c15-eb77-4a88-aa23-9125367e7991\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " Jul 30 17:38:31.671866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671677 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l678v\" (UniqueName: \"kubernetes.io/projected/efdb1c15-eb77-4a88-aa23-9125367e7991-kube-api-access-l678v\") pod \"efdb1c15-eb77-4a88-aa23-9125367e7991\" (UID: \"efdb1c15-eb77-4a88-aa23-9125367e7991\") " Jul 30 17:38:31.671866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671742 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-util\") pod \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " Jul 30 17:38:31.671866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671773 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-bundle\") pod \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " Jul 30 17:38:31.671866 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671832 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj9gm\" (UniqueName: \"kubernetes.io/projected/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-kube-api-access-nj9gm\") pod \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " Jul 30 17:38:31.672106 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671895 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-bundle\") pod \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\" (UID: \"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9\") " Jul 30 17:38:31.672106 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.671933 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-util\") pod \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\" (UID: \"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc\") " Jul 30 17:38:31.672888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.672561 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-bundle" (OuterVolumeSpecName: "bundle") pod "efdb1c15-eb77-4a88-aa23-9125367e7991" (UID: "efdb1c15-eb77-4a88-aa23-9125367e7991"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:31.672888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.672567 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-bundle" (OuterVolumeSpecName: "bundle") pod "5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" (UID: "5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:31.673172 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.673044 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-bundle" (OuterVolumeSpecName: "bundle") pod "e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" (UID: "e87acff2-b3aa-4e46-a89b-4a48ae2f13d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:31.674209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.674185 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efdb1c15-eb77-4a88-aa23-9125367e7991-kube-api-access-l678v" (OuterVolumeSpecName: "kube-api-access-l678v") pod "efdb1c15-eb77-4a88-aa23-9125367e7991" (UID: "efdb1c15-eb77-4a88-aa23-9125367e7991"). InnerVolumeSpecName "kube-api-access-l678v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:38:31.674298 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.674219 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-kube-api-access-hg7hq" (OuterVolumeSpecName: "kube-api-access-hg7hq") pod "e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" (UID: "e87acff2-b3aa-4e46-a89b-4a48ae2f13d9"). InnerVolumeSpecName "kube-api-access-hg7hq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:38:31.674786 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.674764 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-kube-api-access-nj9gm" (OuterVolumeSpecName: "kube-api-access-nj9gm") pod "5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" (UID: "5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc"). InnerVolumeSpecName "kube-api-access-nj9gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:38:31.679049 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.679013 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-util" (OuterVolumeSpecName: "util") pod "e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" (UID: "e87acff2-b3aa-4e46-a89b-4a48ae2f13d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:31.680326 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.680306 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-util" (OuterVolumeSpecName: "util") pod "efdb1c15-eb77-4a88-aa23-9125367e7991" (UID: "efdb1c15-eb77-4a88-aa23-9125367e7991"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:31.680684 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.680668 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-util" (OuterVolumeSpecName: "util") pod "5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" (UID: "5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:38:31.772662 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772632 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg7hq\" (UniqueName: \"kubernetes.io/projected/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-kube-api-access-hg7hq\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772662 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772657 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772662 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772666 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l678v\" (UniqueName: \"kubernetes.io/projected/efdb1c15-eb77-4a88-aa23-9125367e7991-kube-api-access-l678v\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772676 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772684 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772692 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj9gm\" (UniqueName: \"kubernetes.io/projected/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-kube-api-access-nj9gm\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772700 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e87acff2-b3aa-4e46-a89b-4a48ae2f13d9-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772707 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:31.772888 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:31.772714 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efdb1c15-eb77-4a88-aa23-9125367e7991-util\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:38:32.408738 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.408707 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" Jul 30 17:38:32.408738 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.408712 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7" event={"ID":"e87acff2-b3aa-4e46-a89b-4a48ae2f13d9","Type":"ContainerDied","Data":"338279861f43eec931bc22127b627e67c46bf1f8e90c2676c4062e4bae0954d9"} Jul 30 17:38:32.409217 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.408764 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="338279861f43eec931bc22127b627e67c46bf1f8e90c2676c4062e4bae0954d9" Jul 30 17:38:32.410666 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.410643 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" Jul 30 17:38:32.410784 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.410641 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf" event={"ID":"5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc","Type":"ContainerDied","Data":"253350bb4c44c34b1664c057b8f75cefe7dc877d8d90962c9586e17f997e7b97"} Jul 30 17:38:32.410887 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.410794 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="253350bb4c44c34b1664c057b8f75cefe7dc877d8d90962c9586e17f997e7b97" Jul 30 17:38:32.412825 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.412782 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" Jul 30 17:38:32.412937 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.412779 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m" event={"ID":"efdb1c15-eb77-4a88-aa23-9125367e7991","Type":"ContainerDied","Data":"827bf1b061aaf40a725002c61f435f2e5f6506cf6dfd713184a307f504b51b39"} Jul 30 17:38:32.412937 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:32.412931 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="827bf1b061aaf40a725002c61f435f2e5f6506cf6dfd713184a307f504b51b39" Jul 30 17:38:49.958764 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:38:49.958686 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bfb8fbbfd-dvrqx"] Jul 30 17:39:14.985080 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:14.985009 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bfb8fbbfd-dvrqx" podUID="b8657402-f4d7-47f8-8a35-af4a8b7e0c70" containerName="console" containerID="cri-o://72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d" gracePeriod=15 Jul 30 17:39:15.233796 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.233770 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:39:15.331656 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331572 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-oauth-config\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.331656 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331615 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jxs8\" (UniqueName: \"kubernetes.io/projected/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-kube-api-access-6jxs8\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.331656 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331642 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-config\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.331994 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331662 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-oauth-serving-cert\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.331994 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331799 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-serving-cert\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.331994 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331877 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-service-ca\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.331994 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.331915 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-trusted-ca-bundle\") pod \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\" (UID: \"b8657402-f4d7-47f8-8a35-af4a8b7e0c70\") " Jul 30 17:39:15.332200 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.332121 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:39:15.332200 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.332114 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-config" (OuterVolumeSpecName: "console-config") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:39:15.332297 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.332214 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-config\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.332297 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.332231 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-oauth-serving-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.332297 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.332250 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:39:15.332653 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.332629 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 17:39:15.334016 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.333989 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:39:15.334122 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.334068 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-kube-api-access-6jxs8" (OuterVolumeSpecName: "kube-api-access-6jxs8") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "kube-api-access-6jxs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:39:15.334171 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.334120 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8657402-f4d7-47f8-8a35-af4a8b7e0c70" (UID: "b8657402-f4d7-47f8-8a35-af4a8b7e0c70"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:39:15.432685 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.432655 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jxs8\" (UniqueName: \"kubernetes.io/projected/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-kube-api-access-6jxs8\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.432685 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.432681 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-serving-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.432902 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.432695 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-service-ca\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.432902 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.432707 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-trusted-ca-bundle\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.432902 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.432719 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8657402-f4d7-47f8-8a35-af4a8b7e0c70-console-oauth-config\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:39:15.621313 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.621279 2550 generic.go:356] "Generic (PLEG): container finished" podID="b8657402-f4d7-47f8-8a35-af4a8b7e0c70" containerID="72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d" exitCode=0 Jul 30 17:39:15.621485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.621338 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfb8fbbfd-dvrqx" Jul 30 17:39:15.621485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.621366 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfb8fbbfd-dvrqx" event={"ID":"b8657402-f4d7-47f8-8a35-af4a8b7e0c70","Type":"ContainerDied","Data":"72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d"} Jul 30 17:39:15.621485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.621435 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfb8fbbfd-dvrqx" event={"ID":"b8657402-f4d7-47f8-8a35-af4a8b7e0c70","Type":"ContainerDied","Data":"207f3d88e70b1b018916c9244b88ec9ad4d2fd36edd4219e8adb279f98e59e60"} Jul 30 17:39:15.621485 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.621455 2550 scope.go:117] "RemoveContainer" containerID="72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d" Jul 30 17:39:15.630947 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.630933 2550 scope.go:117] "RemoveContainer" containerID="72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d" Jul 30 17:39:15.631177 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:39:15.631161 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d\": container with ID starting with 72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d not found: ID does not exist" containerID="72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d" Jul 30 17:39:15.631250 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.631183 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d"} err="failed to get container status \"72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d\": rpc error: code = NotFound desc = could not find container \"72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d\": container with ID starting with 72d7a43bdde5b1e629fb701fd117a18557e498d4d90384948ded6cdcd6d0f83d not found: ID does not exist" Jul 30 17:39:15.648371 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.648335 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bfb8fbbfd-dvrqx"] Jul 30 17:39:15.654652 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.654630 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bfb8fbbfd-dvrqx"] Jul 30 17:39:15.807868 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:15.807840 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8657402-f4d7-47f8-8a35-af4a8b7e0c70" path="/var/lib/kubelet/pods/b8657402-f4d7-47f8-8a35-af4a8b7e0c70/volumes" Jul 30 17:39:26.379299 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379269 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp"] Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379561 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="extract" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379573 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="extract" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379585 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="util" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379591 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="util" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379599 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="util" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379604 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="util" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379611 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="pull" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379616 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="pull" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379621 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="pull" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379626 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="pull" Jul 30 17:39:26.379647 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379650 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379656 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379663 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379668 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379677 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="util" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379682 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="util" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379688 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="pull" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379692 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="pull" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379699 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="pull" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379704 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="pull" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379712 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8657402-f4d7-47f8-8a35-af4a8b7e0c70" containerName="console" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379717 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8657402-f4d7-47f8-8a35-af4a8b7e0c70" containerName="console" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379723 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="util" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379728 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="util" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379735 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379740 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379788 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="efdb1c15-eb77-4a88-aa23-9125367e7991" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379795 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8657402-f4d7-47f8-8a35-af4a8b7e0c70" containerName="console" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379805 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e87acff2-b3aa-4e46-a89b-4a48ae2f13d9" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379829 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc" containerName="extract" Jul 30 17:39:26.380008 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.379838 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d495923-6d40-4598-b619-679fd72ff675" containerName="extract" Jul 30 17:39:26.384348 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.384328 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.388096 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.388077 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ccdlk\"" Jul 30 17:39:26.397297 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.397276 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp"] Jul 30 17:39:26.413882 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.413861 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.413983 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.413894 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc6dg\" (UniqueName: \"kubernetes.io/projected/910ebae8-59cf-41b7-b3d0-0099badb3ecf-kube-api-access-qc6dg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.413983 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.413913 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.413983 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.413930 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.413983 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.413969 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.414266 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.414108 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.414266 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.414140 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.414266 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.414166 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.414266 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.414206 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515072 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515041 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515072 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515072 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515254 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515089 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515254 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515126 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515254 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515183 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515254 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515210 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc6dg\" (UniqueName: \"kubernetes.io/projected/910ebae8-59cf-41b7-b3d0-0099badb3ecf-kube-api-access-qc6dg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515254 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515233 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515254 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515254 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515561 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515275 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515561 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515439 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515561 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515485 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515878 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515856 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.515967 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.515892 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.516033 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.516017 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.517709 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.517681 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.517828 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.517740 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.525451 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.525415 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/910ebae8-59cf-41b7-b3d0-0099badb3ecf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.526229 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.526212 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc6dg\" (UniqueName: \"kubernetes.io/projected/910ebae8-59cf-41b7-b3d0-0099badb3ecf-kube-api-access-qc6dg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9s4rp\" (UID: \"910ebae8-59cf-41b7-b3d0-0099badb3ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.694546 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.694472 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:26.833615 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.833592 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp"] Jul 30 17:39:26.836022 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:39:26.835992 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910ebae8_59cf_41b7_b3d0_0099badb3ecf.slice/crio-dbfebbf9f7d008343360b269ed1966763e8d4cc1645d58c840b79fffe5808026 WatchSource:0}: Error finding container dbfebbf9f7d008343360b269ed1966763e8d4cc1645d58c840b79fffe5808026: Status 404 returned error can't find the container with id dbfebbf9f7d008343360b269ed1966763e8d4cc1645d58c840b79fffe5808026 Jul 30 17:39:26.837984 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.837943 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:39:26.838094 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.838017 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:39:26.838094 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:26.838061 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:39:27.683244 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:27.683209 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" event={"ID":"910ebae8-59cf-41b7-b3d0-0099badb3ecf","Type":"ContainerStarted","Data":"7979741422c5a90d50b01372831a41d556c7e0f5a0ecaaebbdf4f58182034d90"} Jul 30 17:39:27.683598 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:27.683291 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" event={"ID":"910ebae8-59cf-41b7-b3d0-0099badb3ecf","Type":"ContainerStarted","Data":"dbfebbf9f7d008343360b269ed1966763e8d4cc1645d58c840b79fffe5808026"} Jul 30 17:39:27.695192 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:27.695173 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:27.699874 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:27.699854 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:27.714934 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:27.714895 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" podStartSLOduration=1.714881479 podStartE2EDuration="1.714881479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:27.713433807 +0000 UTC m=+762.445948494" watchObservedRunningTime="2026-07-30 17:39:27.714881479 +0000 UTC m=+762.447396161" Jul 30 17:39:28.688217 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:28.688185 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:39:28.689211 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:39:28.689192 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:40:10.396411 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.396320 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59fc4686ff-kkrkq"] Jul 30 17:40:10.399649 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.399622 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.402876 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.402849 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n5rvf\"" Jul 30 17:40:10.402996 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.402933 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 30 17:40:10.403749 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.403732 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 30 17:40:10.413505 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.413480 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-kkrkq"] Jul 30 17:40:10.490494 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.490459 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-webhook-cert\") pod \"maas-controller-59fc4686ff-kkrkq\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.490494 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.490495 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctpq2\" (UniqueName: \"kubernetes.io/projected/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-kube-api-access-ctpq2\") pod \"maas-controller-59fc4686ff-kkrkq\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.591052 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.591022 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-webhook-cert\") pod \"maas-controller-59fc4686ff-kkrkq\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.591212 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.591057 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctpq2\" (UniqueName: \"kubernetes.io/projected/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-kube-api-access-ctpq2\") pod \"maas-controller-59fc4686ff-kkrkq\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.593594 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.593565 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-webhook-cert\") pod \"maas-controller-59fc4686ff-kkrkq\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.599776 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.599751 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctpq2\" (UniqueName: \"kubernetes.io/projected/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-kube-api-access-ctpq2\") pod \"maas-controller-59fc4686ff-kkrkq\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.710293 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.710168 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:10.851277 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.848200 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-kkrkq"] Jul 30 17:40:10.851942 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:40:10.851917 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b4b9a6_92e8_4d03_aec6_fefdb7a96a83.slice/crio-8dcde8f6eb3a325e3e1882bc408b1dbf373f293edf1c21419086fe2026585a04 WatchSource:0}: Error finding container 8dcde8f6eb3a325e3e1882bc408b1dbf373f293edf1c21419086fe2026585a04: Status 404 returned error can't find the container with id 8dcde8f6eb3a325e3e1882bc408b1dbf373f293edf1c21419086fe2026585a04 Jul 30 17:40:10.892192 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:10.892158 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" event={"ID":"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83","Type":"ContainerStarted","Data":"8dcde8f6eb3a325e3e1882bc408b1dbf373f293edf1c21419086fe2026585a04"} Jul 30 17:40:13.912040 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:13.912002 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" event={"ID":"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83","Type":"ContainerStarted","Data":"0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0"} Jul 30 17:40:13.912448 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:13.912141 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:13.935759 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:13.935714 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" podStartSLOduration=1.6867505440000001 podStartE2EDuration="3.935698131s" totalImagesPullingTime="2.248947587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:13.932864186 +0000 UTC m=+808.665378869" watchObservedRunningTime="2026-07-30 17:40:13.935698131 +0000 UTC m=+808.668212842" Jul 30 17:40:24.919242 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:24.919207 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:40:47.166335 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.166292 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp"] Jul 30 17:40:47.171195 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.171167 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.174049 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.174025 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 30 17:40:47.174167 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.174077 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7bbrz\"" Jul 30 17:40:47.179309 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.179290 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp"] Jul 30 17:40:47.302167 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.302133 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b71deb2-8228-41f5-b81f-af3fde07e727-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-s5wsp\" (UID: \"3b71deb2-8228-41f5-b81f-af3fde07e727\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.302357 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.302195 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptf9\" (UniqueName: \"kubernetes.io/projected/3b71deb2-8228-41f5-b81f-af3fde07e727-kube-api-access-wptf9\") pod \"payload-pre-processing-687dbb6cf6-s5wsp\" (UID: \"3b71deb2-8228-41f5-b81f-af3fde07e727\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.403269 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.403234 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b71deb2-8228-41f5-b81f-af3fde07e727-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-s5wsp\" (UID: \"3b71deb2-8228-41f5-b81f-af3fde07e727\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.403430 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.403283 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptf9\" (UniqueName: \"kubernetes.io/projected/3b71deb2-8228-41f5-b81f-af3fde07e727-kube-api-access-wptf9\") pod \"payload-pre-processing-687dbb6cf6-s5wsp\" (UID: \"3b71deb2-8228-41f5-b81f-af3fde07e727\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.403947 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.403929 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b71deb2-8228-41f5-b81f-af3fde07e727-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-s5wsp\" (UID: \"3b71deb2-8228-41f5-b81f-af3fde07e727\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.414168 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.414141 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptf9\" (UniqueName: \"kubernetes.io/projected/3b71deb2-8228-41f5-b81f-af3fde07e727-kube-api-access-wptf9\") pod \"payload-pre-processing-687dbb6cf6-s5wsp\" (UID: \"3b71deb2-8228-41f5-b81f-af3fde07e727\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.482144 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.482070 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:47.634865 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:47.634805 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp"] Jul 30 17:40:47.636217 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:40:47.636189 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b71deb2_8228_41f5_b81f_af3fde07e727.slice/crio-5235b5a464535617a357c28c0e6fc53290dd98699835991897ab62a54fdf0a68 WatchSource:0}: Error finding container 5235b5a464535617a357c28c0e6fc53290dd98699835991897ab62a54fdf0a68: Status 404 returned error can't find the container with id 5235b5a464535617a357c28c0e6fc53290dd98699835991897ab62a54fdf0a68 Jul 30 17:40:48.086336 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:48.086300 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" event={"ID":"3b71deb2-8228-41f5-b81f-af3fde07e727","Type":"ContainerStarted","Data":"5235b5a464535617a357c28c0e6fc53290dd98699835991897ab62a54fdf0a68"} Jul 30 17:40:50.098462 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:50.098430 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" event={"ID":"3b71deb2-8228-41f5-b81f-af3fde07e727","Type":"ContainerStarted","Data":"288ace16bbda74925f2cd967742a43eceb6f31b5d21f6fa8915ab0b7513c2992"} Jul 30 17:40:50.099070 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:50.099053 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:40:50.116192 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:50.116143 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" podStartSLOduration=1.153549315 podStartE2EDuration="3.116128135s" totalImagesPullingTime="1.96257882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:50.114445325 +0000 UTC m=+844.846960012" watchObservedRunningTime="2026-07-30 17:40:50.116128135 +0000 UTC m=+844.848642817" Jul 30 17:40:55.128725 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:55.128693 2550 generic.go:356] "Generic (PLEG): container finished" podID="910ebae8-59cf-41b7-b3d0-0099badb3ecf" containerID="7979741422c5a90d50b01372831a41d556c7e0f5a0ecaaebbdf4f58182034d90" exitCode=0 Jul 30 17:40:55.129168 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:55.128775 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" event={"ID":"910ebae8-59cf-41b7-b3d0-0099badb3ecf","Type":"ContainerDied","Data":"7979741422c5a90d50b01372831a41d556c7e0f5a0ecaaebbdf4f58182034d90"} Jul 30 17:40:55.129223 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:55.129207 2550 scope.go:117] "RemoveContainer" containerID="7979741422c5a90d50b01372831a41d556c7e0f5a0ecaaebbdf4f58182034d90" Jul 30 17:40:55.130166 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:55.130139 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:40:55.130241 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:55.130225 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:40:55.130291 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:55.130275 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 17:40:56.139721 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:56.139673 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" event={"ID":"910ebae8-59cf-41b7-b3d0-0099badb3ecf","Type":"ContainerStarted","Data":"515d919d3b683acc39e4c5a57ef19f7d2bf31cfc3f3b96aa0e52de74d8bd7a55"} Jul 30 17:40:56.695148 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:56.695115 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:40:56.696244 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:56.696206 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9s4rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 30 17:40:56.696361 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:56.696268 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" podUID="910ebae8-59cf-41b7-b3d0-0099badb3ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 30 17:40:57.695467 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:57.695430 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9s4rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 30 17:40:57.695884 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:57.695496 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" podUID="910ebae8-59cf-41b7-b3d0-0099badb3ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 30 17:40:58.688221 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:58.688188 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:40:58.694794 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:58.694769 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9s4rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 30 17:40:58.694943 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:58.694850 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" podUID="910ebae8-59cf-41b7-b3d0-0099badb3ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 30 17:40:59.695051 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:59.695015 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9s4rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 30 17:40:59.695526 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:40:59.695090 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" podUID="910ebae8-59cf-41b7-b3d0-0099badb3ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 30 17:41:00.698499 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:00.698471 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:41:00.699444 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:00.699427 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9s4rp" Jul 30 17:41:02.113672 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:02.113645 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-s5wsp" Jul 30 17:41:21.111394 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.111360 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4"] Jul 30 17:41:21.115135 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.115113 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.117948 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.117927 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lr279\"" Jul 30 17:41:21.118073 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.117929 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 30 17:41:21.118073 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.117978 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 30 17:41:21.118946 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.118929 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 30 17:41:21.127313 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.127289 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4"] Jul 30 17:41:21.204536 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.204498 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r226\" (UniqueName: \"kubernetes.io/projected/d406272b-e819-4a2a-9aff-40e12d0819ed-kube-api-access-8r226\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.204705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.204566 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.204705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.204600 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d406272b-e819-4a2a-9aff-40e12d0819ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.204705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.204620 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.204705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.204658 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.204705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.204702 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.305957 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.305924 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306126 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.305971 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d406272b-e819-4a2a-9aff-40e12d0819ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306126 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.305993 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306208 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.306122 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306208 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.306175 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306275 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.306236 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r226\" (UniqueName: \"kubernetes.io/projected/d406272b-e819-4a2a-9aff-40e12d0819ed-kube-api-access-8r226\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306385 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.306366 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306446 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.306427 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.306554 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.306537 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.308630 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.308608 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d406272b-e819-4a2a-9aff-40e12d0819ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.308746 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.308653 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d406272b-e819-4a2a-9aff-40e12d0819ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.315435 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.315407 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r226\" (UniqueName: \"kubernetes.io/projected/d406272b-e819-4a2a-9aff-40e12d0819ed-kube-api-access-8r226\") pod \"e2e-trlp-test-simulated-kserve-598466548d-45pp4\" (UID: \"d406272b-e819-4a2a-9aff-40e12d0819ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.426055 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.425969 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:21.573694 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:21.573663 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4"] Jul 30 17:41:21.575438 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:41:21.575413 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd406272b_e819_4a2a_9aff_40e12d0819ed.slice/crio-25e4ed7ddeb24798972d573b077c1653934c2a9387c9c4feb574b5aafe0433df WatchSource:0}: Error finding container 25e4ed7ddeb24798972d573b077c1653934c2a9387c9c4feb574b5aafe0433df: Status 404 returned error can't find the container with id 25e4ed7ddeb24798972d573b077c1653934c2a9387c9c4feb574b5aafe0433df Jul 30 17:41:22.272794 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:22.272751 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" event={"ID":"d406272b-e819-4a2a-9aff-40e12d0819ed","Type":"ContainerStarted","Data":"25e4ed7ddeb24798972d573b077c1653934c2a9387c9c4feb574b5aafe0433df"} Jul 30 17:41:23.280675 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:23.280621 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" event={"ID":"d406272b-e819-4a2a-9aff-40e12d0819ed","Type":"ContainerStarted","Data":"f98adee02393b171f612e1c2ecf2b485aaecff1ac395574a63096d7b2a2debc9"} Jul 30 17:41:23.298388 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:23.298338 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" podStartSLOduration=1.282446731 podStartE2EDuration="2.298322822s" totalImagesPullingTime="1.015876091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:41:23.297356139 +0000 UTC m=+878.029870824" watchObservedRunningTime="2026-07-30 17:41:23.298322822 +0000 UTC m=+878.030837538" Jul 30 17:41:31.426472 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:31.426437 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:31.426991 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:31.426500 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:31.438624 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:31.438602 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:32.337705 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:32.337672 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-45pp4" Jul 30 17:41:36.828418 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.828387 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg"] Jul 30 17:41:36.831998 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.831980 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:36.834982 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.834958 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 30 17:41:36.840945 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.840921 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg"] Jul 30 17:41:36.947909 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.947869 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:36.948098 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.947935 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:36.948098 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.948044 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:36.948098 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.948073 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:36.948251 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.948101 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtcg7\" (UniqueName: \"kubernetes.io/projected/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-kube-api-access-qtcg7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:36.948251 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:36.948142 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049006 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.048922 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049006 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.048958 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049006 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.048980 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcg7\" (UniqueName: \"kubernetes.io/projected/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-kube-api-access-qtcg7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049006 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.049005 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049361 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.049051 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049361 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.049084 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049461 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.049422 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049525 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.049477 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.049525 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.049507 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.051476 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.051455 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.051606 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.051590 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.057287 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.057267 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtcg7\" (UniqueName: \"kubernetes.io/projected/9ce943b5-2bf0-4468-bf83-1fa68a9fc87d-kube-api-access-qtcg7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sqldg\" (UID: \"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.163429 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.163395 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:37.300507 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.300480 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg"] Jul 30 17:41:37.301865 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:41:37.301804 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce943b5_2bf0_4468_bf83_1fa68a9fc87d.slice/crio-091b769d2be1aeb333c41469e8a882b8e2e7fcb390db6a944ab02a3d52a28b58 WatchSource:0}: Error finding container 091b769d2be1aeb333c41469e8a882b8e2e7fcb390db6a944ab02a3d52a28b58: Status 404 returned error can't find the container with id 091b769d2be1aeb333c41469e8a882b8e2e7fcb390db6a944ab02a3d52a28b58 Jul 30 17:41:37.362460 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:37.362424 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" event={"ID":"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d","Type":"ContainerStarted","Data":"091b769d2be1aeb333c41469e8a882b8e2e7fcb390db6a944ab02a3d52a28b58"} Jul 30 17:41:38.368797 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:38.368761 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" event={"ID":"9ce943b5-2bf0-4468-bf83-1fa68a9fc87d","Type":"ContainerStarted","Data":"243e2c71bb0d8d70927bca48fb7fd52701008ab08062d55814b06a9dcf0122ca"} Jul 30 17:41:38.388039 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:38.387985 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" podStartSLOduration=2.191020729 podStartE2EDuration="2.387969663s" totalImagesPullingTime="196.948934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:41:38.385335712 +0000 UTC m=+893.117850397" watchObservedRunningTime="2026-07-30 17:41:38.387969663 +0000 UTC m=+893.120484383" Jul 30 17:41:47.163908 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:47.163850 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:47.164331 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:47.163952 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:47.176780 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:47.176756 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:41:47.426789 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:41:47.426709 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sqldg" Jul 30 17:44:11.770880 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:11.770841 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-kkrkq"] Jul 30 17:44:11.771484 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:11.771064 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" podUID="56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" containerName="manager" containerID="cri-o://0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0" gracePeriod=30 Jul 30 17:44:12.022331 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.022276 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:44:12.120830 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.120780 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctpq2\" (UniqueName: \"kubernetes.io/projected/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-kube-api-access-ctpq2\") pod \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " Jul 30 17:44:12.120987 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.120887 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-webhook-cert\") pod \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\" (UID: \"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83\") " Jul 30 17:44:12.123059 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.123028 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-kube-api-access-ctpq2" (OuterVolumeSpecName: "kube-api-access-ctpq2") pod "56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" (UID: "56b4b9a6-92e8-4d03-aec6-fefdb7a96a83"). InnerVolumeSpecName "kube-api-access-ctpq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:44:12.123059 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.123047 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" (UID: "56b4b9a6-92e8-4d03-aec6-fefdb7a96a83"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:44:12.161379 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.161346 2550 generic.go:356] "Generic (PLEG): container finished" podID="56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" containerID="0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0" exitCode=0 Jul 30 17:44:12.161525 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.161395 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" Jul 30 17:44:12.161525 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.161397 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" event={"ID":"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83","Type":"ContainerDied","Data":"0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0"} Jul 30 17:44:12.161634 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.161522 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-kkrkq" event={"ID":"56b4b9a6-92e8-4d03-aec6-fefdb7a96a83","Type":"ContainerDied","Data":"8dcde8f6eb3a325e3e1882bc408b1dbf373f293edf1c21419086fe2026585a04"} Jul 30 17:44:12.161634 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.161550 2550 scope.go:117] "RemoveContainer" containerID="0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0" Jul 30 17:44:12.170416 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.170399 2550 scope.go:117] "RemoveContainer" containerID="0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0" Jul 30 17:44:12.170663 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:44:12.170645 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0\": container with ID starting with 0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0 not found: ID does not exist" containerID="0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0" Jul 30 17:44:12.170725 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.170670 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0"} err="failed to get container status \"0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0\": rpc error: code = NotFound desc = could not find container \"0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0\": container with ID starting with 0ab8cbea13e7de7666101950c94944a1d91f1323949ba58085c8a0816ac712d0 not found: ID does not exist" Jul 30 17:44:12.183403 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.183378 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-kkrkq"] Jul 30 17:44:12.187945 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.187924 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-kkrkq"] Jul 30 17:44:12.222046 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.222018 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctpq2\" (UniqueName: \"kubernetes.io/projected/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-kube-api-access-ctpq2\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:44:12.222046 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:12.222042 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83-webhook-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:44:13.810885 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:44:13.810846 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" path="/var/lib/kubelet/pods/56b4b9a6-92e8-4d03-aec6-fefdb7a96a83/volumes" Jul 30 17:57:10.463457 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.463422 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr"] Jul 30 17:57:10.463949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.463736 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" containerName="manager" Jul 30 17:57:10.463949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.463746 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" containerName="manager" Jul 30 17:57:10.463949 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.463858 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="56b4b9a6-92e8-4d03-aec6-fefdb7a96a83" containerName="manager" Jul 30 17:57:10.466902 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.466886 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.474146 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.474129 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-215b0c63-kserve-self-signed-certs\"" Jul 30 17:57:10.485484 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.485453 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr"] Jul 30 17:57:10.592059 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.592034 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-home\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.592209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.592067 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-dshm\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.592209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.592086 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6g44\" (UniqueName: \"kubernetes.io/projected/4aacfddf-5d74-48e4-b8d2-987222f9fb44-kube-api-access-k6g44\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.592209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.592124 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tls-certs\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.592209 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.592167 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tmp-dir\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.592341 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.592218 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-model-cache\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693190 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693157 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tls-certs\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693207 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tmp-dir\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693245 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-model-cache\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693273 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-home\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693291 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-dshm\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693343 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693316 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6g44\" (UniqueName: \"kubernetes.io/projected/4aacfddf-5d74-48e4-b8d2-987222f9fb44-kube-api-access-k6g44\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693735 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693707 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-home\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.693735 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693730 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-model-cache\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.694112 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.693806 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tmp-dir\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.695607 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.695587 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-dshm\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.695698 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.695662 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tls-certs\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.701481 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.701459 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6g44\" (UniqueName: \"kubernetes.io/projected/4aacfddf-5d74-48e4-b8d2-987222f9fb44-kube-api-access-k6g44\") pod \"e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.775678 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.775628 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:10.923474 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:10.923445 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr"] Jul 30 17:57:10.925443 ip-10-0-140-246 kubenswrapper[2550]: W0730 17:57:10.925416 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4aacfddf_5d74_48e4_b8d2_987222f9fb44.slice/crio-5616ef1a258b741986c835f4c4f1106d6a670e2886e2d14624da08bb1aff052c WatchSource:0}: Error finding container 5616ef1a258b741986c835f4c4f1106d6a670e2886e2d14624da08bb1aff052c: Status 404 returned error can't find the container with id 5616ef1a258b741986c835f4c4f1106d6a670e2886e2d14624da08bb1aff052c Jul 30 17:57:11.091449 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:11.091409 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" event={"ID":"4aacfddf-5d74-48e4-b8d2-987222f9fb44","Type":"ContainerStarted","Data":"fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf"} Jul 30 17:57:11.091600 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:11.091454 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" event={"ID":"4aacfddf-5d74-48e4-b8d2-987222f9fb44","Type":"ContainerStarted","Data":"5616ef1a258b741986c835f4c4f1106d6a670e2886e2d14624da08bb1aff052c"} Jul 30 17:57:11.119683 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:11.119638 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" podStartSLOduration=1.119624117 podStartE2EDuration="1.119624117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:57:11.119254494 +0000 UTC m=+1825.851769178" watchObservedRunningTime="2026-07-30 17:57:11.119624117 +0000 UTC m=+1825.852138800" Jul 30 17:57:20.776642 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:20.776600 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:20.777200 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:20.776654 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:20.789057 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:20.789026 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:21.156690 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:21.156658 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:43.223428 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:43.223393 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr"] Jul 30 17:57:58.242920 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.242879 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" podUID="4aacfddf-5d74-48e4-b8d2-987222f9fb44" containerName="main" containerID="cri-o://fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf" gracePeriod=45 Jul 30 17:57:58.494168 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.494106 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:58.579668 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.579631 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-dshm\") pod \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " Jul 30 17:57:58.579853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.579692 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tls-certs\") pod \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " Jul 30 17:57:58.579853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.579710 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-model-cache\") pod \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " Jul 30 17:57:58.579853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.579776 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tmp-dir\") pod \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " Jul 30 17:57:58.579853 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.579804 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-home\") pod \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " Jul 30 17:57:58.580072 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.579874 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6g44\" (UniqueName: \"kubernetes.io/projected/4aacfddf-5d74-48e4-b8d2-987222f9fb44-kube-api-access-k6g44\") pod \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\" (UID: \"4aacfddf-5d74-48e4-b8d2-987222f9fb44\") " Jul 30 17:57:58.580072 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.580050 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-model-cache" (OuterVolumeSpecName: "model-cache") pod "4aacfddf-5d74-48e4-b8d2-987222f9fb44" (UID: "4aacfddf-5d74-48e4-b8d2-987222f9fb44"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:57:58.580186 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.580065 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4aacfddf-5d74-48e4-b8d2-987222f9fb44" (UID: "4aacfddf-5d74-48e4-b8d2-987222f9fb44"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:57:58.580186 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.580152 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-home" (OuterVolumeSpecName: "home") pod "4aacfddf-5d74-48e4-b8d2-987222f9fb44" (UID: "4aacfddf-5d74-48e4-b8d2-987222f9fb44"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:57:58.580186 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.580179 2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tmp-dir\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:57:58.580302 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.580199 2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-model-cache\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:57:58.582118 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.582093 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aacfddf-5d74-48e4-b8d2-987222f9fb44-kube-api-access-k6g44" (OuterVolumeSpecName: "kube-api-access-k6g44") pod "4aacfddf-5d74-48e4-b8d2-987222f9fb44" (UID: "4aacfddf-5d74-48e4-b8d2-987222f9fb44"). InnerVolumeSpecName "kube-api-access-k6g44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 17:57:58.582203 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.582152 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4aacfddf-5d74-48e4-b8d2-987222f9fb44" (UID: "4aacfddf-5d74-48e4-b8d2-987222f9fb44"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 17:57:58.582203 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.582152 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-dshm" (OuterVolumeSpecName: "dshm") pod "4aacfddf-5d74-48e4-b8d2-987222f9fb44" (UID: "4aacfddf-5d74-48e4-b8d2-987222f9fb44"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 17:57:58.681540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.681496 2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-home\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:57:58.681540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.681536 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6g44\" (UniqueName: \"kubernetes.io/projected/4aacfddf-5d74-48e4-b8d2-987222f9fb44-kube-api-access-k6g44\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:57:58.681540 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.681550 2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4aacfddf-5d74-48e4-b8d2-987222f9fb44-dshm\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:57:58.682039 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:58.681563 2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4aacfddf-5d74-48e4-b8d2-987222f9fb44-tls-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 17:57:59.335861 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.335803 2550 generic.go:356] "Generic (PLEG): container finished" podID="4aacfddf-5d74-48e4-b8d2-987222f9fb44" containerID="fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf" exitCode=0 Jul 30 17:57:59.336235 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.335877 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" Jul 30 17:57:59.336235 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.335886 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" event={"ID":"4aacfddf-5d74-48e4-b8d2-987222f9fb44","Type":"ContainerDied","Data":"fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf"} Jul 30 17:57:59.336235 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.335924 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr" event={"ID":"4aacfddf-5d74-48e4-b8d2-987222f9fb44","Type":"ContainerDied","Data":"5616ef1a258b741986c835f4c4f1106d6a670e2886e2d14624da08bb1aff052c"} Jul 30 17:57:59.336235 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.335946 2550 scope.go:117] "RemoveContainer" containerID="fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf" Jul 30 17:57:59.345527 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.345511 2550 scope.go:117] "RemoveContainer" containerID="fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf" Jul 30 17:57:59.345764 ip-10-0-140-246 kubenswrapper[2550]: E0730 17:57:59.345745 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf\": container with ID starting with fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf not found: ID does not exist" containerID="fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf" Jul 30 17:57:59.345852 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.345771 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf"} err="failed to get container status \"fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf\": rpc error: code = NotFound desc = could not find container \"fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf\": container with ID starting with fcdddf25fade46325392546c01e495789dd481f1ed59044cbc9c5fc14d9483cf not found: ID does not exist" Jul 30 17:57:59.358419 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.358392 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr"] Jul 30 17:57:59.363876 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.363855 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-215b0c63-kserve-6c495c9847-7r6hr"] Jul 30 17:57:59.807969 ip-10-0-140-246 kubenswrapper[2550]: I0730 17:57:59.807939 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aacfddf-5d74-48e4-b8d2-987222f9fb44" path="/var/lib/kubelet/pods/4aacfddf-5d74-48e4-b8d2-987222f9fb44/volumes" Jul 30 18:02:26.223148 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.223107 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw"] Jul 30 18:02:26.223966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.223624 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aacfddf-5d74-48e4-b8d2-987222f9fb44" containerName="main" Jul 30 18:02:26.223966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.223638 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aacfddf-5d74-48e4-b8d2-987222f9fb44" containerName="main" Jul 30 18:02:26.223966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.223721 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aacfddf-5d74-48e4-b8d2-987222f9fb44" containerName="main" Jul 30 18:02:26.227038 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.227020 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.230698 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.230666 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8a48bf8b-openshift-default-dockercfg-gx2cs\"" Jul 30 18:02:26.249956 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.249935 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw"] Jul 30 18:02:26.347086 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347049 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-socket\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347086 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347083 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-certs\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347333 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347105 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-data\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347333 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347155 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-envoy\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347333 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347241 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-credential-socket\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347333 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347288 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-token\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347333 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347304 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lksfh\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-kube-api-access-lksfh\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347333 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347326 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-podinfo\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.347701 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.347340 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istiod-ca-cert\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.448789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448758 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-socket\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.448789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448786 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-certs\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448807 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-data\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448854 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-envoy\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448900 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-credential-socket\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448917 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-token\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448939 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lksfh\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-kube-api-access-lksfh\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448970 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-podinfo\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.448996 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istiod-ca-cert\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449456 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.449206 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-certs\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449456 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.449301 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-data\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449456 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.449443 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-credential-socket\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449621 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.449514 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-socket\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.449715 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.449698 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istiod-ca-cert\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.451396 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.451377 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-envoy\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.451639 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.451622 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-podinfo\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.460365 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.460334 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-token\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.461415 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.461392 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lksfh\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-kube-api-access-lksfh\") pod \"e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.539966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.539914 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:26.681202 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.681178 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw"] Jul 30 18:02:26.683023 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:02:26.682962 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0845dc81_0a88_40f4_b9ec_b123d1b87a53.slice/crio-506df0e8fba6e38c87fe3a94bd898f775e02cc95d8ebcf24bd38f8c52f9485d1 WatchSource:0}: Error finding container 506df0e8fba6e38c87fe3a94bd898f775e02cc95d8ebcf24bd38f8c52f9485d1: Status 404 returned error can't find the container with id 506df0e8fba6e38c87fe3a94bd898f775e02cc95d8ebcf24bd38f8c52f9485d1 Jul 30 18:02:26.685855 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.685800 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:02:26.686017 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.685897 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:02:26.686017 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.685931 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:02:26.691733 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:26.691704 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" event={"ID":"0845dc81-0a88-40f4-b9ec-b123d1b87a53","Type":"ContainerStarted","Data":"506df0e8fba6e38c87fe3a94bd898f775e02cc95d8ebcf24bd38f8c52f9485d1"} Jul 30 18:02:27.698778 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:27.698741 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" event={"ID":"0845dc81-0a88-40f4-b9ec-b123d1b87a53","Type":"ContainerStarted","Data":"457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99"} Jul 30 18:02:27.722612 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:27.722564 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podStartSLOduration=1.722549004 podStartE2EDuration="1.722549004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:02:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:02:27.720963195 +0000 UTC m=+2142.453477880" watchObservedRunningTime="2026-07-30 18:02:27.722549004 +0000 UTC m=+2142.455063687" Jul 30 18:02:28.540688 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:28.540655 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:28.545259 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:28.545238 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:28.704879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:28.704848 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:28.705621 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:28.705604 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:44.746971 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:44.746928 2550 patch_prober.go:28] interesting pod/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 18:02:44.747447 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:44.747005 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 18:02:45.802280 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:45.802192 2550 generic.go:356] "Generic (PLEG): container finished" podID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerID="457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99" exitCode=0 Jul 30 18:02:45.807720 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:45.807687 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" event={"ID":"0845dc81-0a88-40f4-b9ec-b123d1b87a53","Type":"ContainerDied","Data":"457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99"} Jul 30 18:02:45.808074 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:45.808060 2550 scope.go:117] "RemoveContainer" containerID="457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99" Jul 30 18:02:45.809063 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:45.809026 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:02:45.809210 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:45.809132 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:02:45.809210 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:45.809182 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:02:46.808580 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:46.808548 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" event={"ID":"0845dc81-0a88-40f4-b9ec-b123d1b87a53","Type":"ContainerStarted","Data":"4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427"} Jul 30 18:02:47.541026 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:47.540991 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:47.541463 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:47.541343 2550 patch_prober.go:28] interesting pod/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 18:02:47.541625 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:47.541603 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 18:02:48.541110 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:48.541057 2550 patch_prober.go:28] interesting pod/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 18:02:48.541641 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:48.541154 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 18:02:49.540703 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:49.540660 2550 patch_prober.go:28] interesting pod/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 18:02:49.540920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:49.540737 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 18:02:50.541168 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:50.541107 2550 patch_prober.go:28] interesting pod/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 18:02:50.541566 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:50.541193 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 18:02:51.543865 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:51.543840 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:51.544237 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:51.544210 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:02:51.544738 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:02:51.544725 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:03:11.069548 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:11.069514 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw"] Jul 30 18:03:11.070047 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:11.069786 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" containerID="cri-o://4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427" gracePeriod=30 Jul 30 18:03:12.093136 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.093098 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4"] Jul 30 18:03:12.096486 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.096470 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.099238 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.099218 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fee844af-gw-openshift-default-dockercfg-lwzk2\"" Jul 30 18:03:12.114055 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.114034 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4"] Jul 30 18:03:12.235185 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235148 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-credential-socket\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235185 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235181 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-envoy\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235390 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235203 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-podinfo\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235390 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235220 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcv9m\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-kube-api-access-mcv9m\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235390 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235324 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-socket\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235390 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235371 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-data\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235528 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235411 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93453264-bfc7-4fe4-970d-27e92cf6741f-istiod-ca-cert\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235528 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235430 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-token\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.235528 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.235460 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-certs\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336596 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336567 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-podinfo\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336596 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336599 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv9m\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-kube-api-access-mcv9m\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336866 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336647 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-socket\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336866 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-data\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336866 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93453264-bfc7-4fe4-970d-27e92cf6741f-istiod-ca-cert\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336866 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336733 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-token\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.336866 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336833 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-certs\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337166 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336911 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-credential-socket\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337166 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.336935 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-envoy\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337249 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.337232 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-data\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.337296 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-certs\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337376 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.337360 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-socket\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337465 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.337451 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-credential-socket\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.337568 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.337538 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93453264-bfc7-4fe4-970d-27e92cf6741f-istiod-ca-cert\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.339406 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.339380 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-envoy\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.339712 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.339693 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-podinfo\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.345225 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.345171 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-token\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.345298 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.345278 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcv9m\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-kube-api-access-mcv9m\") pod \"e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.406373 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.406346 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:12.543638 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.543617 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4"] Jul 30 18:03:12.545390 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:03:12.545364 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93453264_bfc7_4fe4_970d_27e92cf6741f.slice/crio-5b1f97f60ddf0501afd4011e2279b2dde52c3baa84efdb10676a352bb2ba78c6 WatchSource:0}: Error finding container 5b1f97f60ddf0501afd4011e2279b2dde52c3baa84efdb10676a352bb2ba78c6: Status 404 returned error can't find the container with id 5b1f97f60ddf0501afd4011e2279b2dde52c3baa84efdb10676a352bb2ba78c6 Jul 30 18:03:12.547384 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.547347 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:12.547444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.547419 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:12.547480 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.547445 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:12.933605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.933514 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" event={"ID":"93453264-bfc7-4fe4-970d-27e92cf6741f","Type":"ContainerStarted","Data":"e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273"} Jul 30 18:03:12.933605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.933573 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" event={"ID":"93453264-bfc7-4fe4-970d-27e92cf6741f","Type":"ContainerStarted","Data":"5b1f97f60ddf0501afd4011e2279b2dde52c3baa84efdb10676a352bb2ba78c6"} Jul 30 18:03:12.954101 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:12.954050 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podStartSLOduration=0.954029307 podStartE2EDuration="954.029307ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:12.952448592 +0000 UTC m=+2187.684963274" watchObservedRunningTime="2026-07-30 18:03:12.954029307 +0000 UTC m=+2187.686543992" Jul 30 18:03:13.407580 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:13.407545 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:13.412054 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:13.412034 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:13.938492 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:13.938452 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:13.939585 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:13.939559 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:16.315271 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.315249 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:03:16.473433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473403 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-certs\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473436 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istiod-ca-cert\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473654 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473462 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-credential-socket\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473654 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473493 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-token\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473654 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473527 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-socket\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473654 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473579 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-envoy\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473654 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473611 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-podinfo\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473636 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:16.473932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473656 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-data\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473728 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lksfh\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-kube-api-access-lksfh\") pod \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\" (UID: \"0845dc81-0a88-40f4-b9ec-b123d1b87a53\") " Jul 30 18:03:16.473932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473786 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:16.473932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473783 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:03:16.473932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.473805 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:16.474287 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.474092 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.474287 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.474111 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.474287 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.474125 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.474287 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.474139 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.474526 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.474504 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-data" (OuterVolumeSpecName: "istio-data") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:16.475914 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.475895 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-token" (OuterVolumeSpecName: "istio-token") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:03:16.475985 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.475968 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:03:16.476299 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.476278 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:16.476362 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.476310 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-kube-api-access-lksfh" (OuterVolumeSpecName: "kube-api-access-lksfh") pod "0845dc81-0a88-40f4-b9ec-b123d1b87a53" (UID: "0845dc81-0a88-40f4-b9ec-b123d1b87a53"). InnerVolumeSpecName "kube-api-access-lksfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:03:16.575481 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.575452 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.575481 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.575477 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.575481 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.575486 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.575683 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.575494 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0845dc81-0a88-40f4-b9ec-b123d1b87a53-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.575683 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.575502 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lksfh\" (UniqueName: \"kubernetes.io/projected/0845dc81-0a88-40f4-b9ec-b123d1b87a53-kube-api-access-lksfh\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:16.955595 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.955564 2550 generic.go:356] "Generic (PLEG): container finished" podID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerID="4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427" exitCode=0 Jul 30 18:03:16.955741 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.955602 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" Jul 30 18:03:16.955741 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.955648 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" event={"ID":"0845dc81-0a88-40f4-b9ec-b123d1b87a53","Type":"ContainerDied","Data":"4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427"} Jul 30 18:03:16.955741 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.955708 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw" event={"ID":"0845dc81-0a88-40f4-b9ec-b123d1b87a53","Type":"ContainerDied","Data":"506df0e8fba6e38c87fe3a94bd898f775e02cc95d8ebcf24bd38f8c52f9485d1"} Jul 30 18:03:16.955741 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.955735 2550 scope.go:117] "RemoveContainer" containerID="4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427" Jul 30 18:03:16.965520 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.965503 2550 scope.go:117] "RemoveContainer" containerID="457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99" Jul 30 18:03:16.973624 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.973607 2550 scope.go:117] "RemoveContainer" containerID="4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427" Jul 30 18:03:16.973856 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:03:16.973838 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427\": container with ID starting with 4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427 not found: ID does not exist" containerID="4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427" Jul 30 18:03:16.973928 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.973869 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427"} err="failed to get container status \"4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427\": rpc error: code = NotFound desc = could not find container \"4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427\": container with ID starting with 4e76f56527ac826d91db626600f46e2c01d39b5ba6d463e4fc90d7bb40c98427 not found: ID does not exist" Jul 30 18:03:16.973928 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.973889 2550 scope.go:117] "RemoveContainer" containerID="457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99" Jul 30 18:03:16.974144 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:03:16.974128 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99\": container with ID starting with 457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99 not found: ID does not exist" containerID="457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99" Jul 30 18:03:16.974192 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.974149 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99"} err="failed to get container status \"457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99\": rpc error: code = NotFound desc = could not find container \"457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99\": container with ID starting with 457518a2750c8c046b3389faf9b43acc27ca4f7e31cf98d394ff7cfc4abb2f99 not found: ID does not exist" Jul 30 18:03:16.979989 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.979967 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw"] Jul 30 18:03:16.985261 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:16.985234 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8a48bf8b-openshift-default-7f6685d65d-brcnw"] Jul 30 18:03:17.809373 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:17.809339 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" path="/var/lib/kubelet/pods/0845dc81-0a88-40f4-b9ec-b123d1b87a53/volumes" Jul 30 18:03:28.940422 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:28.940365 2550 patch_prober.go:28] interesting pod/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 18:03:28.940764 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:28.940417 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 18:03:29.020159 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:29.020097 2550 generic.go:356] "Generic (PLEG): container finished" podID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerID="e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273" exitCode=0 Jul 30 18:03:29.020458 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:29.020168 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" event={"ID":"93453264-bfc7-4fe4-970d-27e92cf6741f","Type":"ContainerDied","Data":"e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273"} Jul 30 18:03:29.020564 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:29.020549 2550 scope.go:117] "RemoveContainer" containerID="e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273" Jul 30 18:03:29.021554 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:29.021528 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:29.021653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:29.021606 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:29.021717 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:29.021653 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:30.031090 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:30.031047 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" event={"ID":"93453264-bfc7-4fe4-970d-27e92cf6741f","Type":"ContainerStarted","Data":"1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd"} Jul 30 18:03:30.406752 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:30.406715 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:30.406980 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:30.406958 2550 patch_prober.go:28] interesting pod/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 18:03:30.407062 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:30.407012 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 18:03:31.407330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:31.407294 2550 patch_prober.go:28] interesting pod/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 18:03:31.407712 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:31.407344 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 18:03:32.407710 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:32.407679 2550 patch_prober.go:28] interesting pod/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 18:03:32.408135 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:32.407740 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 18:03:33.407079 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:33.407046 2550 patch_prober.go:28] interesting pod/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 18:03:33.407244 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:33.407101 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 18:03:34.410912 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:34.410887 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:34.411296 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:34.411130 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:34.411879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:34.411865 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:46.259785 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:46.259692 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4"] Jul 30 18:03:46.260253 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:46.259951 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" containerID="cri-o://1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd" gracePeriod=30 Jul 30 18:03:47.135099 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135069 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv"] Jul 30 18:03:47.135455 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135427 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" Jul 30 18:03:47.135455 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135446 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" Jul 30 18:03:47.135645 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135490 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" Jul 30 18:03:47.135645 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135498 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" Jul 30 18:03:47.135645 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135583 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" Jul 30 18:03:47.135645 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.135594 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0845dc81-0a88-40f4-b9ec-b123d1b87a53" containerName="istio-proxy" Jul 30 18:03:47.140063 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.140043 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.143630 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.143611 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c2052f19-openshift-default-dockercfg-nsp72\"" Jul 30 18:03:47.162566 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.162539 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv"] Jul 30 18:03:47.235936 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.235907 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-envoy\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.235951 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-certs\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236016 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-credential-socket\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236161 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236117 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-token\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236161 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236154 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b8a83-1d6b-492e-83f7-0a55f8081858-istiod-ca-cert\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236235 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-podinfo\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236316 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236273 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-data\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236372 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236317 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-socket\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.236425 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.236366 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jn77\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-kube-api-access-7jn77\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337439 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337403 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-certs\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337448 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-credential-socket\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337482 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-token\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337513 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b8a83-1d6b-492e-83f7-0a55f8081858-istiod-ca-cert\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337543 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-podinfo\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337567 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-data\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337600 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-socket\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337627 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jn77\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-kube-api-access-7jn77\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.337892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337704 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-envoy\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.338258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337927 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-certs\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.338258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.337957 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-credential-socket\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.338258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.338128 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-data\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.338258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.338152 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-socket\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.338258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.338246 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b8a83-1d6b-492e-83f7-0a55f8081858-istiod-ca-cert\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.340157 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.340140 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-envoy\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.340345 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.340327 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-podinfo\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.348960 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.348938 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jn77\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-kube-api-access-7jn77\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.349407 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.349388 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-token\") pod \"e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.452547 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.452485 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:47.589189 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.589164 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv"] Jul 30 18:03:47.590532 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:03:47.590508 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554b8a83_1d6b_492e_83f7_0a55f8081858.slice/crio-809e71cae731214d66216a69849542f8c60b6f450c6195151512cf71bcca6569 WatchSource:0}: Error finding container 809e71cae731214d66216a69849542f8c60b6f450c6195151512cf71bcca6569: Status 404 returned error can't find the container with id 809e71cae731214d66216a69849542f8c60b6f450c6195151512cf71bcca6569 Jul 30 18:03:47.592617 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.592586 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:47.592709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.592654 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:47.592709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:47.592682 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:03:48.129144 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:48.129103 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" event={"ID":"554b8a83-1d6b-492e-83f7-0a55f8081858","Type":"ContainerStarted","Data":"752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995"} Jul 30 18:03:48.129334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:48.129157 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" event={"ID":"554b8a83-1d6b-492e-83f7-0a55f8081858","Type":"ContainerStarted","Data":"809e71cae731214d66216a69849542f8c60b6f450c6195151512cf71bcca6569"} Jul 30 18:03:48.155299 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:48.155252 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podStartSLOduration=1.155239292 podStartE2EDuration="1.155239292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:48.153691467 +0000 UTC m=+2222.886206161" watchObservedRunningTime="2026-07-30 18:03:48.155239292 +0000 UTC m=+2222.887753975" Jul 30 18:03:48.453232 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:48.453150 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:48.457975 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:48.457954 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:49.134386 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:49.134356 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:49.135172 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:49.135154 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:03:49.414328 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:49.414235 2550 patch_prober.go:28] interesting pod/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 18:03:49.414328 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:49.414304 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 18:03:51.543278 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.543255 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:51.671793 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.671761 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-podinfo\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.671997 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.671805 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93453264-bfc7-4fe4-970d-27e92cf6741f-istiod-ca-cert\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.671995 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-socket\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672044 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-certs\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672064 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-envoy\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672231 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672092 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcv9m\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-kube-api-access-mcv9m\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672231 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672137 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-data\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672231 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672159 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-token\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672231 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672197 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-credential-socket\") pod \"93453264-bfc7-4fe4-970d-27e92cf6741f\" (UID: \"93453264-bfc7-4fe4-970d-27e92cf6741f\") " Jul 30 18:03:51.672429 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672248 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93453264-bfc7-4fe4-970d-27e92cf6741f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:03:51.672429 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672249 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:51.672429 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672283 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:51.672586 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672475 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:51.672586 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672496 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.672586 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672513 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93453264-bfc7-4fe4-970d-27e92cf6741f-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.672586 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672525 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.673019 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.672998 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-data" (OuterVolumeSpecName: "istio-data") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:51.674045 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.674024 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:03:51.674433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.674414 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-kube-api-access-mcv9m" (OuterVolumeSpecName: "kube-api-access-mcv9m") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "kube-api-access-mcv9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:03:51.674681 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.674662 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:03:51.674738 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.674665 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-token" (OuterVolumeSpecName: "istio-token") pod "93453264-bfc7-4fe4-970d-27e92cf6741f" (UID: "93453264-bfc7-4fe4-970d-27e92cf6741f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:03:51.773151 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.773119 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcv9m\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-kube-api-access-mcv9m\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.773151 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.773145 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.773304 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.773157 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.773304 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.773168 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.773304 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.773179 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:51.773304 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:51.773189 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93453264-bfc7-4fe4-970d-27e92cf6741f-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:03:52.150261 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.150226 2550 generic.go:356] "Generic (PLEG): container finished" podID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerID="1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd" exitCode=0 Jul 30 18:03:52.150461 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.150270 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" Jul 30 18:03:52.150461 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.150309 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" event={"ID":"93453264-bfc7-4fe4-970d-27e92cf6741f","Type":"ContainerDied","Data":"1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd"} Jul 30 18:03:52.150461 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.150372 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4" event={"ID":"93453264-bfc7-4fe4-970d-27e92cf6741f","Type":"ContainerDied","Data":"5b1f97f60ddf0501afd4011e2279b2dde52c3baa84efdb10676a352bb2ba78c6"} Jul 30 18:03:52.150461 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.150402 2550 scope.go:117] "RemoveContainer" containerID="1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd" Jul 30 18:03:52.161503 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.161482 2550 scope.go:117] "RemoveContainer" containerID="e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273" Jul 30 18:03:52.170643 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.170626 2550 scope.go:117] "RemoveContainer" containerID="1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd" Jul 30 18:03:52.170934 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:03:52.170916 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd\": container with ID starting with 1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd not found: ID does not exist" containerID="1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd" Jul 30 18:03:52.170993 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.170944 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd"} err="failed to get container status \"1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd\": rpc error: code = NotFound desc = could not find container \"1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd\": container with ID starting with 1089f65281016211213b4a13e19a4a25e519252459b827f57f1f1bb6fd3fc3dd not found: ID does not exist" Jul 30 18:03:52.170993 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.170962 2550 scope.go:117] "RemoveContainer" containerID="e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273" Jul 30 18:03:52.171186 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:03:52.171169 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273\": container with ID starting with e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273 not found: ID does not exist" containerID="e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273" Jul 30 18:03:52.171225 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.171190 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273"} err="failed to get container status \"e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273\": rpc error: code = NotFound desc = could not find container \"e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273\": container with ID starting with e0b52c8d3d44fd8c13815f499a26073504dd760eae06acdc9b144769279ed273 not found: ID does not exist" Jul 30 18:03:52.171966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.171946 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4"] Jul 30 18:03:52.174880 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:52.174859 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fee844af-gw-openshift-default-5866c986f6-4vdp4"] Jul 30 18:03:53.808538 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:03:53.808504 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" path="/var/lib/kubelet/pods/93453264-bfc7-4fe4-970d-27e92cf6741f/volumes" Jul 30 18:04:04.136345 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.136314 2550 patch_prober.go:28] interesting pod/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 30 18:04:04.136678 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.136380 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 30 18:04:04.214229 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.214198 2550 generic.go:356] "Generic (PLEG): container finished" podID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerID="752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995" exitCode=0 Jul 30 18:04:04.214368 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.214260 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" event={"ID":"554b8a83-1d6b-492e-83f7-0a55f8081858","Type":"ContainerDied","Data":"752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995"} Jul 30 18:04:04.214642 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.214625 2550 scope.go:117] "RemoveContainer" containerID="752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995" Jul 30 18:04:04.215626 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.215602 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:04.215701 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.215684 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:04.215753 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:04.215727 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:05.221641 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:05.221594 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" event={"ID":"554b8a83-1d6b-492e-83f7-0a55f8081858","Type":"ContainerStarted","Data":"69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5"} Jul 30 18:04:05.452973 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:05.452935 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:04:05.453268 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:05.453244 2550 patch_prober.go:28] interesting pod/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 30 18:04:05.453377 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:05.453303 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 30 18:04:06.453452 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:06.453414 2550 patch_prober.go:28] interesting pod/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 30 18:04:06.453881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:06.453466 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 30 18:04:07.453402 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:07.453318 2550 patch_prober.go:28] interesting pod/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 30 18:04:07.453402 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:07.453378 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 30 18:04:08.453541 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:08.453505 2550 patch_prober.go:28] interesting pod/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 30 18:04:08.453967 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:08.453563 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 30 18:04:09.456892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:09.456863 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:04:09.457284 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:09.457136 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:04:09.457766 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:09.457749 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:04:23.065449 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.065419 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv"] Jul 30 18:04:23.065997 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.065618 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" containerID="cri-o://69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5" gracePeriod=30 Jul 30 18:04:23.823956 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.823918 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr"] Jul 30 18:04:23.824270 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.824257 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" Jul 30 18:04:23.824317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.824273 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" Jul 30 18:04:23.824317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.824281 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" Jul 30 18:04:23.824317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.824287 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" Jul 30 18:04:23.824425 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.824355 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" Jul 30 18:04:23.826595 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.826581 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.829390 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.829370 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9995abfe-openshift-default-dockercfg-52qtn\"" Jul 30 18:04:23.843897 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.843831 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr"] Jul 30 18:04:23.950837 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.950784 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-envoy\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951018 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.950984 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-certs\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951078 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951015 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92bc2\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-kube-api-access-92bc2\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951078 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951059 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-data\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951192 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951087 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-socket\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951192 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951124 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-token\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951192 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951179 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-podinfo\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951328 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951206 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-credential-socket\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:23.951328 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:23.951240 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istiod-ca-cert\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052148 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052112 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-envoy\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052342 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052178 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-certs\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052342 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92bc2\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-kube-api-access-92bc2\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052342 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052214 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-data\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052342 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052233 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-socket\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052396 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-token\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052465 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-podinfo\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052492 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-credential-socket\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052523 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istiod-ca-cert\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052593 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-certs\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052794 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052765 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-data\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052878 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052804 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-socket\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.052998 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.052976 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-credential-socket\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.060107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.053850 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istiod-ca-cert\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.060107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.055568 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-envoy\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.060107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.055941 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-podinfo\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.062995 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.062968 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-token\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.063125 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.063004 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92bc2\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-kube-api-access-92bc2\") pod \"e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.137570 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.137545 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:24.286999 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.286970 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr"] Jul 30 18:04:24.288128 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:04:24.288101 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b487de8_767f_4f60_b3ca_6a3b0fd98d40.slice/crio-8ba84613437b69b753d12c6da1bac326f4f645cf90ecad84c234f39a2b26a860 WatchSource:0}: Error finding container 8ba84613437b69b753d12c6da1bac326f4f645cf90ecad84c234f39a2b26a860: Status 404 returned error can't find the container with id 8ba84613437b69b753d12c6da1bac326f4f645cf90ecad84c234f39a2b26a860 Jul 30 18:04:24.290306 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.290277 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:24.290384 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.290338 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:24.290384 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.290366 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:24.331877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.331847 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" event={"ID":"9b487de8-767f-4f60-b3ca-6a3b0fd98d40","Type":"ContainerStarted","Data":"8ba84613437b69b753d12c6da1bac326f4f645cf90ecad84c234f39a2b26a860"} Jul 30 18:04:24.459339 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.459259 2550 patch_prober.go:28] interesting pod/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 18:04:24.459467 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:24.459328 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 18:04:25.338532 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:25.338493 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" event={"ID":"9b487de8-767f-4f60-b3ca-6a3b0fd98d40","Type":"ContainerStarted","Data":"77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc"} Jul 30 18:04:25.362034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:25.361984 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podStartSLOduration=2.36196802 podStartE2EDuration="2.36196802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:04:25.358538765 +0000 UTC m=+2260.091053448" watchObservedRunningTime="2026-07-30 18:04:25.36196802 +0000 UTC m=+2260.094482704" Jul 30 18:04:26.138363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:26.138329 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:26.143142 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:26.143116 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:26.343500 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:26.343471 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:26.344448 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:26.344433 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:28.331384 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.331326 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:04:28.356284 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.356257 2550 generic.go:356] "Generic (PLEG): container finished" podID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerID="69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5" exitCode=0 Jul 30 18:04:28.356439 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.356299 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" Jul 30 18:04:28.356439 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.356341 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" event={"ID":"554b8a83-1d6b-492e-83f7-0a55f8081858","Type":"ContainerDied","Data":"69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5"} Jul 30 18:04:28.356439 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.356392 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv" event={"ID":"554b8a83-1d6b-492e-83f7-0a55f8081858","Type":"ContainerDied","Data":"809e71cae731214d66216a69849542f8c60b6f450c6195151512cf71bcca6569"} Jul 30 18:04:28.356439 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.356417 2550 scope.go:117] "RemoveContainer" containerID="69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5" Jul 30 18:04:28.366850 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.366830 2550 scope.go:117] "RemoveContainer" containerID="752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995" Jul 30 18:04:28.376934 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.376913 2550 scope.go:117] "RemoveContainer" containerID="69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5" Jul 30 18:04:28.377255 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:04:28.377235 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5\": container with ID starting with 69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5 not found: ID does not exist" containerID="69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5" Jul 30 18:04:28.377355 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.377262 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5"} err="failed to get container status \"69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5\": rpc error: code = NotFound desc = could not find container \"69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5\": container with ID starting with 69ea4f4d6cb0cfd7505f20f332c0ce0c29af1e92a4aa13b9cb48cac6d59fe3c5 not found: ID does not exist" Jul 30 18:04:28.377355 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.377280 2550 scope.go:117] "RemoveContainer" containerID="752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995" Jul 30 18:04:28.377572 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:04:28.377553 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995\": container with ID starting with 752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995 not found: ID does not exist" containerID="752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995" Jul 30 18:04:28.377628 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.377582 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995"} err="failed to get container status \"752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995\": rpc error: code = NotFound desc = could not find container \"752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995\": container with ID starting with 752d3caafaf79c60a80c7a648e6305555cf28126c8a203c94219351ea1892995 not found: ID does not exist" Jul 30 18:04:28.389634 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.389609 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b8a83-1d6b-492e-83f7-0a55f8081858-istiod-ca-cert\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.389776 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.389649 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-certs\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.389776 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.389669 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-envoy\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.389776 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.389700 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-socket\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.390003 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.389981 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:04:28.390064 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.390014 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:04:28.390064 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.390030 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554b8a83-1d6b-492e-83f7-0a55f8081858-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:04:28.392009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.391989 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:04:28.491104 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491073 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-credential-socket\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.491264 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491113 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-data\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.491264 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491137 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jn77\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-kube-api-access-7jn77\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.491264 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491169 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-token\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.491264 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491198 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-podinfo\") pod \"554b8a83-1d6b-492e-83f7-0a55f8081858\" (UID: \"554b8a83-1d6b-492e-83f7-0a55f8081858\") " Jul 30 18:04:28.491448 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491357 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:04:28.491688 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491657 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.491688 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491682 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554b8a83-1d6b-492e-83f7-0a55f8081858-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.491884 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491695 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.491884 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491708 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.491884 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.491722 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.492084 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.492063 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-data" (OuterVolumeSpecName: "istio-data") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:04:28.493996 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.493969 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:04:28.493996 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.493975 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-kube-api-access-7jn77" (OuterVolumeSpecName: "kube-api-access-7jn77") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "kube-api-access-7jn77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:04:28.494154 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.493975 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-token" (OuterVolumeSpecName: "istio-token") pod "554b8a83-1d6b-492e-83f7-0a55f8081858" (UID: "554b8a83-1d6b-492e-83f7-0a55f8081858"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:04:28.593007 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.592954 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jn77\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-kube-api-access-7jn77\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.593007 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.592995 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.593007 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.593009 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.593300 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.593020 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554b8a83-1d6b-492e-83f7-0a55f8081858-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:04:28.680243 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.680213 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv"] Jul 30 18:04:28.683604 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:28.683579 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c2052f19-openshift-default-699669b68d-cgdzv"] Jul 30 18:04:29.807510 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:29.807478 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" path="/var/lib/kubelet/pods/554b8a83-1d6b-492e-83f7-0a55f8081858/volumes" Jul 30 18:04:39.416023 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:39.415989 2550 generic.go:356] "Generic (PLEG): container finished" podID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerID="77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc" exitCode=0 Jul 30 18:04:39.416023 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:39.416026 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" event={"ID":"9b487de8-767f-4f60-b3ca-6a3b0fd98d40","Type":"ContainerDied","Data":"77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc"} Jul 30 18:04:39.416479 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:39.416403 2550 scope.go:117] "RemoveContainer" containerID="77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc" Jul 30 18:04:39.417418 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:39.417391 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:39.417476 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:39.417462 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:39.417519 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:39.417494 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:04:40.424073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:40.424025 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" event={"ID":"9b487de8-767f-4f60-b3ca-6a3b0fd98d40","Type":"ContainerStarted","Data":"c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675"} Jul 30 18:04:41.138690 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:41.138657 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:41.140133 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:41.140099 2550 patch_prober.go:28] interesting pod/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 18:04:41.140266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:41.140162 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 18:04:41.344465 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:41.344421 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:42.138246 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:42.138206 2550 patch_prober.go:28] interesting pod/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 18:04:42.138799 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:42.138770 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 18:04:43.137980 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:43.137941 2550 patch_prober.go:28] interesting pod/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 18:04:43.138200 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:43.138012 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 18:04:44.139637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:44.138942 2550 patch_prober.go:28] interesting pod/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 18:04:44.139637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:44.139002 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 18:04:45.142752 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:45.142708 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:45.143688 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:45.143665 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:04:58.750124 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:58.750089 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr"] Jul 30 18:04:58.750645 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:04:58.750326 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" containerID="cri-o://c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675" gracePeriod=30 Jul 30 18:05:00.145656 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:00.145619 2550 patch_prober.go:28] interesting pod/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 18:05:00.146066 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:00.145675 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 18:05:04.005848 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.005803 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:05:04.111291 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111262 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-token\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111306 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-credential-socket\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111333 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-certs\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111355 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-envoy\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111377 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92bc2\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-kube-api-access-92bc2\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111416 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-data\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111438 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-socket\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111471 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111465 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-podinfo\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111938 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111486 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istiod-ca-cert\") pod \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\" (UID: \"9b487de8-767f-4f60-b3ca-6a3b0fd98d40\") " Jul 30 18:05:04.111938 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111611 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:05:04.111938 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111704 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:05:04.111938 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111798 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:05:04.111938 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111930 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:05:04.112158 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111948 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.112158 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111968 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.112158 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.111982 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.112562 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.112530 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:05:04.113673 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.113644 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:05:04.113887 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.113868 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:05:04.113887 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.113874 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:05:04.113977 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.113886 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-kube-api-access-92bc2" (OuterVolumeSpecName: "kube-api-access-92bc2") pod "9b487de8-767f-4f60-b3ca-6a3b0fd98d40" (UID: "9b487de8-767f-4f60-b3ca-6a3b0fd98d40"). InnerVolumeSpecName "kube-api-access-92bc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:05:04.213037 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.213002 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.213037 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.213033 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92bc2\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-kube-api-access-92bc2\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.213037 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.213042 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.213037 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.213051 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.213037 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.213059 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.213380 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.213068 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b487de8-767f-4f60-b3ca-6a3b0fd98d40-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:05:04.562164 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.562075 2550 generic.go:356] "Generic (PLEG): container finished" podID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerID="c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675" exitCode=0 Jul 30 18:05:04.562164 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.562114 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" event={"ID":"9b487de8-767f-4f60-b3ca-6a3b0fd98d40","Type":"ContainerDied","Data":"c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675"} Jul 30 18:05:04.562164 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.562126 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" Jul 30 18:05:04.562164 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.562159 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr" event={"ID":"9b487de8-767f-4f60-b3ca-6a3b0fd98d40","Type":"ContainerDied","Data":"8ba84613437b69b753d12c6da1bac326f4f645cf90ecad84c234f39a2b26a860"} Jul 30 18:05:04.562448 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.562182 2550 scope.go:117] "RemoveContainer" containerID="c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675" Jul 30 18:05:04.572695 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.572677 2550 scope.go:117] "RemoveContainer" containerID="77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc" Jul 30 18:05:04.580976 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.580958 2550 scope.go:117] "RemoveContainer" containerID="c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675" Jul 30 18:05:04.581258 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:05:04.581240 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675\": container with ID starting with c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675 not found: ID does not exist" containerID="c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675" Jul 30 18:05:04.581334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.581265 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675"} err="failed to get container status \"c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675\": rpc error: code = NotFound desc = could not find container \"c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675\": container with ID starting with c6ec6cf454a281e544719436a28cd342252024b4cbc76238143ce30baf424675 not found: ID does not exist" Jul 30 18:05:04.581334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.581281 2550 scope.go:117] "RemoveContainer" containerID="77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc" Jul 30 18:05:04.581531 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:05:04.581514 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc\": container with ID starting with 77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc not found: ID does not exist" containerID="77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc" Jul 30 18:05:04.581572 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.581537 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc"} err="failed to get container status \"77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc\": rpc error: code = NotFound desc = could not find container \"77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc\": container with ID starting with 77e5cd151851f6016e5272f49a2d4ecba80c69cc31f317e363cb094810f90ecc not found: ID does not exist" Jul 30 18:05:04.586557 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.586536 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr"] Jul 30 18:05:04.591447 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:04.591427 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9995abfe-openshift-default-66ccfc4c5b-q8zhr"] Jul 30 18:05:05.807752 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:05:05.807711 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" path="/var/lib/kubelet/pods/9b487de8-767f-4f60-b3ca-6a3b0fd98d40/volumes" Jul 30 18:07:09.321734 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.321644 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b"] Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322032 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322044 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322059 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322065 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322071 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322076 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322085 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322090 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322146 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322154 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" Jul 30 18:07:09.322262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.322161 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="93453264-bfc7-4fe4-970d-27e92cf6741f" containerName="istio-proxy" Jul 30 18:07:09.324105 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.324084 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.327260 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.327231 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e004b9c2-openshift-default-dockercfg-bpq7v\"" Jul 30 18:07:09.351913 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.351882 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b"] Jul 30 18:07:09.465177 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465126 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-socket\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465394 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465204 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-envoy\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465394 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465249 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dc2t\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-kube-api-access-6dc2t\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465394 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465377 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-podinfo\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465601 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465432 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istiod-ca-cert\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465601 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465471 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-token\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465601 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465565 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-certs\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465601 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465598 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-data\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.465877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.465628 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-credential-socket\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566459 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566424 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-certs\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566459 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566461 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-data\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566482 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-credential-socket\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566518 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-socket\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566553 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-envoy\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566587 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dc2t\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-kube-api-access-6dc2t\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566653 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-podinfo\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566689 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istiod-ca-cert\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.566744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-token\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.567107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566960 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-credential-socket\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.567107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.566975 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-certs\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.567107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.567039 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-socket\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.567256 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.567234 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-data\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.567558 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.567503 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istiod-ca-cert\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.569098 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.569077 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-envoy\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.569422 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.569403 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-podinfo\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.575069 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.575013 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-token\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.575151 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.575135 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dc2t\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-kube-api-access-6dc2t\") pod \"e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.634432 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.634391 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:09.780468 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.780437 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b"] Jul 30 18:07:09.781627 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:07:09.781603 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f719f2_72e4_44a2_83ba_0a3f20da12aa.slice/crio-1409d693eee68059e7c32dfdaf8799a0c9b092a2db5141188cb2416d59b986e0 WatchSource:0}: Error finding container 1409d693eee68059e7c32dfdaf8799a0c9b092a2db5141188cb2416d59b986e0: Status 404 returned error can't find the container with id 1409d693eee68059e7c32dfdaf8799a0c9b092a2db5141188cb2416d59b986e0 Jul 30 18:07:09.783915 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.783877 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:07:09.783996 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.783957 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:07:09.784039 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:09.783998 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:07:10.186286 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:10.186245 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" event={"ID":"a3f719f2-72e4-44a2-83ba-0a3f20da12aa","Type":"ContainerStarted","Data":"c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3"} Jul 30 18:07:10.186466 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:10.186314 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" event={"ID":"a3f719f2-72e4-44a2-83ba-0a3f20da12aa","Type":"ContainerStarted","Data":"1409d693eee68059e7c32dfdaf8799a0c9b092a2db5141188cb2416d59b986e0"} Jul 30 18:07:10.211996 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:10.211949 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podStartSLOduration=1.2119317569999999 podStartE2EDuration="1.211931757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:07:10.21000275 +0000 UTC m=+2424.942517437" watchObservedRunningTime="2026-07-30 18:07:10.211931757 +0000 UTC m=+2424.944446439" Jul 30 18:07:10.635168 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:10.635128 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:10.640475 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:10.640450 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:11.191262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:11.191229 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:11.192256 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:11.192238 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:13.988647 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:13.988609 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn"] Jul 30 18:07:13.989095 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:13.989080 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b487de8-767f-4f60-b3ca-6a3b0fd98d40" containerName="istio-proxy" Jul 30 18:07:13.989153 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:13.989098 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="554b8a83-1d6b-492e-83f7-0a55f8081858" containerName="istio-proxy" Jul 30 18:07:13.991145 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:13.991130 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:13.993857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:13.993833 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e004b9c2\"" Jul 30 18:07:13.994294 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:13.994276 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e004b9c2-dockercfg-9ptj8\"" Jul 30 18:07:14.002346 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.002306 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn"] Jul 30 18:07:14.113532 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.113498 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.113705 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.113648 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vll2\" (UniqueName: \"kubernetes.io/projected/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-kube-api-access-5vll2\") pod \"payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.214064 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.214036 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vll2\" (UniqueName: \"kubernetes.io/projected/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-kube-api-access-5vll2\") pod \"payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.214227 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.214077 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.214703 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.214684 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.223406 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.223381 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vll2\" (UniqueName: \"kubernetes.io/projected/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-kube-api-access-5vll2\") pod \"payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.301925 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.301868 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:14.433949 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:14.433924 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn"] Jul 30 18:07:14.435504 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:07:14.435459 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b7dce75_cd2b_4576_a11f_b26a1a0fb601.slice/crio-a508feee6216a9abdf6b3eee72ce6a5c07eea492f585a04d1c357d7536596c58 WatchSource:0}: Error finding container a508feee6216a9abdf6b3eee72ce6a5c07eea492f585a04d1c357d7536596c58: Status 404 returned error can't find the container with id a508feee6216a9abdf6b3eee72ce6a5c07eea492f585a04d1c357d7536596c58 Jul 30 18:07:15.213304 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:15.213267 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" event={"ID":"3b7dce75-cd2b-4576-a11f-b26a1a0fb601","Type":"ContainerStarted","Data":"44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3"} Jul 30 18:07:15.213719 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:15.213333 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" event={"ID":"3b7dce75-cd2b-4576-a11f-b26a1a0fb601","Type":"ContainerStarted","Data":"a508feee6216a9abdf6b3eee72ce6a5c07eea492f585a04d1c357d7536596c58"} Jul 30 18:07:15.213719 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:15.213367 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:15.235633 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:15.235590 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" podStartSLOduration=2.2355765 podStartE2EDuration="2.2355765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:07:15.234418438 +0000 UTC m=+2429.966933119" watchObservedRunningTime="2026-07-30 18:07:15.2355765 +0000 UTC m=+2429.968091182" Jul 30 18:07:26.221028 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:26.220994 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:30.289415 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:30.289377 2550 generic.go:356] "Generic (PLEG): container finished" podID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerID="c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3" exitCode=0 Jul 30 18:07:30.289842 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:30.289451 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" event={"ID":"a3f719f2-72e4-44a2-83ba-0a3f20da12aa","Type":"ContainerDied","Data":"c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3"} Jul 30 18:07:30.289892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:30.289849 2550 scope.go:117] "RemoveContainer" containerID="c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3" Jul 30 18:07:30.290788 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:30.290757 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:07:30.290921 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:30.290876 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:07:30.290987 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:30.290929 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:07:31.296137 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:31.296105 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" event={"ID":"a3f719f2-72e4-44a2-83ba-0a3f20da12aa","Type":"ContainerStarted","Data":"851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8"} Jul 30 18:07:31.635234 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:31.635189 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:31.635498 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:31.635466 2550 patch_prober.go:28] interesting pod/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 18:07:31.635592 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:31.635530 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 18:07:32.635402 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:32.635360 2550 patch_prober.go:28] interesting pod/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 18:07:32.635934 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:32.635434 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 18:07:33.635686 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:33.635647 2550 patch_prober.go:28] interesting pod/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 18:07:33.636546 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:33.636515 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 18:07:34.635764 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:34.635716 2550 patch_prober.go:28] interesting pod/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 18:07:34.636270 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:34.635785 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 18:07:35.638433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:35.638403 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:35.638899 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:35.638847 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:35.639440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:35.639425 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:07:55.377167 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.377131 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn"] Jul 30 18:07:55.377805 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.377372 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" podUID="3b7dce75-cd2b-4576-a11f-b26a1a0fb601" containerName="payload-pre-processing" containerID="cri-o://44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3" gracePeriod=30 Jul 30 18:07:55.646206 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.646164 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:55.765361 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.765337 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vll2\" (UniqueName: \"kubernetes.io/projected/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-kube-api-access-5vll2\") pod \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " Jul 30 18:07:55.765516 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.765410 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-plugins-config-volume\") pod \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\" (UID: \"3b7dce75-cd2b-4576-a11f-b26a1a0fb601\") " Jul 30 18:07:55.765798 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.765770 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b7dce75-cd2b-4576-a11f-b26a1a0fb601" (UID: "3b7dce75-cd2b-4576-a11f-b26a1a0fb601"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:07:55.767535 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.767510 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-kube-api-access-5vll2" (OuterVolumeSpecName: "kube-api-access-5vll2") pod "3b7dce75-cd2b-4576-a11f-b26a1a0fb601" (UID: "3b7dce75-cd2b-4576-a11f-b26a1a0fb601"). InnerVolumeSpecName "kube-api-access-5vll2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:07:55.866743 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.866711 2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-plugins-config-volume\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:07:55.866743 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:55.866737 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vll2\" (UniqueName: \"kubernetes.io/projected/3b7dce75-cd2b-4576-a11f-b26a1a0fb601-kube-api-access-5vll2\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:07:56.438298 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.438257 2550 generic.go:356] "Generic (PLEG): container finished" podID="3b7dce75-cd2b-4576-a11f-b26a1a0fb601" containerID="44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3" exitCode=0 Jul 30 18:07:56.438298 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.438299 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" Jul 30 18:07:56.438780 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.438336 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" event={"ID":"3b7dce75-cd2b-4576-a11f-b26a1a0fb601","Type":"ContainerDied","Data":"44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3"} Jul 30 18:07:56.438780 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.438394 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn" event={"ID":"3b7dce75-cd2b-4576-a11f-b26a1a0fb601","Type":"ContainerDied","Data":"a508feee6216a9abdf6b3eee72ce6a5c07eea492f585a04d1c357d7536596c58"} Jul 30 18:07:56.438780 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.438418 2550 scope.go:117] "RemoveContainer" containerID="44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3" Jul 30 18:07:56.446970 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.446903 2550 scope.go:117] "RemoveContainer" containerID="44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3" Jul 30 18:07:56.447157 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:07:56.447138 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3\": container with ID starting with 44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3 not found: ID does not exist" containerID="44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3" Jul 30 18:07:56.447223 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.447169 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3"} err="failed to get container status \"44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3\": rpc error: code = NotFound desc = could not find container \"44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3\": container with ID starting with 44430e34d174afcb0bbb61462682e1c64c4c6eccfa5025ada075a35d969b14f3 not found: ID does not exist" Jul 30 18:07:56.456947 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.456920 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn"] Jul 30 18:07:56.464672 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:56.464655 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e004b9c2-64b745dcbd-kqzxn"] Jul 30 18:07:57.807715 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:07:57.807681 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b7dce75-cd2b-4576-a11f-b26a1a0fb601" path="/var/lib/kubelet/pods/3b7dce75-cd2b-4576-a11f-b26a1a0fb601/volumes" Jul 30 18:08:02.647901 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:02.647869 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b"] Jul 30 18:08:02.648350 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:02.648124 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" containerID="cri-o://851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8" gracePeriod=30 Jul 30 18:08:05.640408 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:05.640378 2550 patch_prober.go:28] interesting pod/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 30 18:08:05.640779 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:05.640425 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 18:08:07.903665 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:07.903638 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:08:08.063778 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063714 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-envoy\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.063778 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063752 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-token\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.063963 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063782 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dc2t\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-kube-api-access-6dc2t\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.063963 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063893 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-socket\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.063963 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063927 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-certs\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.063963 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063962 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istiod-ca-cert\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.064169 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.063989 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-podinfo\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.064169 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064017 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-credential-socket\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.064169 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064083 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-data\") pod \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\" (UID: \"a3f719f2-72e4-44a2-83ba-0a3f20da12aa\") " Jul 30 18:08:08.064318 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064179 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:08:08.064318 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064192 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:08:08.064427 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064344 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.064427 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064360 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.064427 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064386 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:08:08.064617 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.064527 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:08:08.065021 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.065000 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:08:08.066041 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.066013 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:08:08.066127 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.066048 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-kube-api-access-6dc2t" (OuterVolumeSpecName: "kube-api-access-6dc2t") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "kube-api-access-6dc2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:08:08.066650 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.066629 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:08:08.066650 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.066635 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3f719f2-72e4-44a2-83ba-0a3f20da12aa" (UID: "a3f719f2-72e4-44a2-83ba-0a3f20da12aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:08:08.165561 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165538 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.165561 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165560 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dc2t\" (UniqueName: \"kubernetes.io/projected/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-kube-api-access-6dc2t\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.165709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165571 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.165709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165579 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.165709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165587 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.165709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165595 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.165709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.165603 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f719f2-72e4-44a2-83ba-0a3f20da12aa-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:08:08.501571 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.501537 2550 generic.go:356] "Generic (PLEG): container finished" podID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerID="851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8" exitCode=0 Jul 30 18:08:08.501747 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.501585 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" Jul 30 18:08:08.501747 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.501629 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" event={"ID":"a3f719f2-72e4-44a2-83ba-0a3f20da12aa","Type":"ContainerDied","Data":"851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8"} Jul 30 18:08:08.501747 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.501683 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b" event={"ID":"a3f719f2-72e4-44a2-83ba-0a3f20da12aa","Type":"ContainerDied","Data":"1409d693eee68059e7c32dfdaf8799a0c9b092a2db5141188cb2416d59b986e0"} Jul 30 18:08:08.501747 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.501709 2550 scope.go:117] "RemoveContainer" containerID="851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8" Jul 30 18:08:08.511055 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.511041 2550 scope.go:117] "RemoveContainer" containerID="c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3" Jul 30 18:08:08.518857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.518840 2550 scope.go:117] "RemoveContainer" containerID="851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8" Jul 30 18:08:08.519087 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:08:08.519070 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8\": container with ID starting with 851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8 not found: ID does not exist" containerID="851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8" Jul 30 18:08:08.519146 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.519095 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8"} err="failed to get container status \"851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8\": rpc error: code = NotFound desc = could not find container \"851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8\": container with ID starting with 851560d31e48e647c85ad2bdf08de788780f2ea826dcb3682ba63612e6a387f8 not found: ID does not exist" Jul 30 18:08:08.519146 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.519110 2550 scope.go:117] "RemoveContainer" containerID="c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3" Jul 30 18:08:08.519305 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:08:08.519290 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3\": container with ID starting with c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3 not found: ID does not exist" containerID="c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3" Jul 30 18:08:08.519341 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.519309 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3"} err="failed to get container status \"c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3\": rpc error: code = NotFound desc = could not find container \"c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3\": container with ID starting with c35f8cad321b694fb6845d3fd1fc2fe5a235c3baae41750e03d0aa02cb13d1a3 not found: ID does not exist" Jul 30 18:08:08.524538 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.524514 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b"] Jul 30 18:08:08.526760 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:08.526740 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e004b9c2-openshift-default-68fdc88864-lf92b"] Jul 30 18:08:09.808316 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:08:09.808283 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" path="/var/lib/kubelet/pods/a3f719f2-72e4-44a2-83ba-0a3f20da12aa/volumes" Jul 30 18:10:19.493071 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493030 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4"] Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493389 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493401 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493421 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7dce75-cd2b-4576-a11f-b26a1a0fb601" containerName="payload-pre-processing" Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493427 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7dce75-cd2b-4576-a11f-b26a1a0fb601" containerName="payload-pre-processing" Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493479 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493486 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" Jul 30 18:10:19.493545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.493492 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7dce75-cd2b-4576-a11f-b26a1a0fb601" containerName="payload-pre-processing" Jul 30 18:10:19.496309 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.496294 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.499100 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.499079 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-657b86df-openshift-default-dockercfg-rk9xt\"" Jul 30 18:10:19.507590 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.507572 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4"] Jul 30 18:10:19.601466 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.601429 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-data\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.601713 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.601695 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-socket\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.601906 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.601872 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-credential-socket\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.601906 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.601904 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-envoy\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.602097 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.601936 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49qg7\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-kube-api-access-49qg7\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.602097 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.601968 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-certs\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.602097 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.602083 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74cc2123-669b-4443-ba18-e694d315477f-istio-podinfo\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.602266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.602107 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-istio-token\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.602266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.602131 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74cc2123-669b-4443-ba18-e694d315477f-istiod-ca-cert\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703000 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.702965 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-credential-socket\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703000 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703001 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-envoy\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703243 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703032 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49qg7\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-kube-api-access-49qg7\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703243 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703057 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-certs\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703243 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703149 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74cc2123-669b-4443-ba18-e694d315477f-istio-podinfo\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703243 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703172 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-istio-token\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703243 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74cc2123-669b-4443-ba18-e694d315477f-istiod-ca-cert\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703509 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703258 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-data\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703509 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703288 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-socket\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703509 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703356 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-credential-socket\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703509 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703441 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-certs\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703699 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703623 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-socket\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.703699 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.703633 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-data\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.704235 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.704211 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74cc2123-669b-4443-ba18-e694d315477f-istiod-ca-cert\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.705509 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.705490 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-envoy\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.705744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.705726 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74cc2123-669b-4443-ba18-e694d315477f-istio-podinfo\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.712750 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.712729 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-istio-token\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.712926 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.712906 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49qg7\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-kube-api-access-49qg7\") pod \"e2e-mt-657b86df-openshift-default-798d58545b-7gnc4\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.807759 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.807664 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:19.945146 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.945117 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4"] Jul 30 18:10:19.946542 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:10:19.946501 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74cc2123_669b_4443_ba18_e694d315477f.slice/crio-1883e2fdd5d746a1b097176a7af5eb038106aa455e609706fdd429476ac27e60 WatchSource:0}: Error finding container 1883e2fdd5d746a1b097176a7af5eb038106aa455e609706fdd429476ac27e60: Status 404 returned error can't find the container with id 1883e2fdd5d746a1b097176a7af5eb038106aa455e609706fdd429476ac27e60 Jul 30 18:10:19.948600 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.948568 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:10:19.948705 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.948625 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:10:19.948705 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:19.948655 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:10:20.178503 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:20.178463 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" event={"ID":"74cc2123-669b-4443-ba18-e694d315477f","Type":"ContainerStarted","Data":"830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e"} Jul 30 18:10:20.178680 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:20.178527 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" event={"ID":"74cc2123-669b-4443-ba18-e694d315477f","Type":"ContainerStarted","Data":"1883e2fdd5d746a1b097176a7af5eb038106aa455e609706fdd429476ac27e60"} Jul 30 18:10:20.205336 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:20.205283 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podStartSLOduration=1.205245342 podStartE2EDuration="1.205245342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:10:20.202563673 +0000 UTC m=+2614.935078355" watchObservedRunningTime="2026-07-30 18:10:20.205245342 +0000 UTC m=+2614.937760026" Jul 30 18:10:20.808420 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:20.808389 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:21.812558 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:21.812532 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:22.188199 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:22.188165 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:22.189148 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:22.189128 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:37.647597 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:37.646914 2550 patch_prober.go:28] interesting pod/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": read tcp 10.132.0.2:35548->10.132.0.48:15021: read: connection reset by peer" start-of-body= Jul 30 18:10:37.647597 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:37.646980 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": read tcp 10.132.0.2:35548->10.132.0.48:15021: read: connection reset by peer" Jul 30 18:10:38.284520 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:38.284489 2550 generic.go:356] "Generic (PLEG): container finished" podID="74cc2123-669b-4443-ba18-e694d315477f" containerID="830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e" exitCode=0 Jul 30 18:10:38.284708 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:38.284568 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" event={"ID":"74cc2123-669b-4443-ba18-e694d315477f","Type":"ContainerDied","Data":"830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e"} Jul 30 18:10:38.284970 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:38.284954 2550 scope.go:117] "RemoveContainer" containerID="830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e" Jul 30 18:10:38.285973 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:38.285945 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:10:38.286078 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:38.286017 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:10:38.286078 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:38.286053 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:10:39.292170 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:39.292133 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" event={"ID":"74cc2123-669b-4443-ba18-e694d315477f","Type":"ContainerStarted","Data":"fbcb4298574bf926db0d3fd4c37ec5cd34cefba0e31a34061c98adb199240ec7"} Jul 30 18:10:39.808012 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:39.807982 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:39.808196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:39.808180 2550 patch_prober.go:28] interesting pod/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 30 18:10:39.808257 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:39.808232 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 30 18:10:40.808657 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:40.808623 2550 patch_prober.go:28] interesting pod/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 30 18:10:40.809059 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:40.808675 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 30 18:10:41.808047 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:41.808010 2550 patch_prober.go:28] interesting pod/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 30 18:10:41.808219 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:41.808064 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 30 18:10:42.808149 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:42.808112 2550 patch_prober.go:28] interesting pod/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 30 18:10:42.808554 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:42.808164 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 30 18:10:43.811520 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:43.811485 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:43.812003 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:43.811874 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:10:43.812538 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:10:43.812519 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:12:29.639745 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:29.639710 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4"] Jul 30 18:12:29.640210 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:29.639967 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" containerID="cri-o://fbcb4298574bf926db0d3fd4c37ec5cd34cefba0e31a34061c98adb199240ec7" gracePeriod=30 Jul 30 18:12:34.879602 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:34.879570 2550 generic.go:356] "Generic (PLEG): container finished" podID="74cc2123-669b-4443-ba18-e694d315477f" containerID="fbcb4298574bf926db0d3fd4c37ec5cd34cefba0e31a34061c98adb199240ec7" exitCode=0 Jul 30 18:12:34.880046 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:34.879592 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" event={"ID":"74cc2123-669b-4443-ba18-e694d315477f","Type":"ContainerDied","Data":"fbcb4298574bf926db0d3fd4c37ec5cd34cefba0e31a34061c98adb199240ec7"} Jul 30 18:12:34.880046 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:34.879673 2550 scope.go:117] "RemoveContainer" containerID="830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e" Jul 30 18:12:34.899148 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:12:34.899090 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e\": container with ID starting with 830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e not found: ID does not exist" containerID="830de60ef97beb337f3e297c318c71091e19c9a30041d07997a96e870c0bcc7e" Jul 30 18:12:34.905359 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:34.905337 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:12:35.023101 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023071 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-certs\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023114 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74cc2123-669b-4443-ba18-e694d315477f-istiod-ca-cert\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023133 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49qg7\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-kube-api-access-49qg7\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023152 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-credential-socket\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023438 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023299 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-socket\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023438 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023360 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-data\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023438 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023393 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74cc2123-669b-4443-ba18-e694d315477f-istio-podinfo\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023438 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023396 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:12:35.023653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023424 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-envoy\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023449 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:12:35.023653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023489 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-istio-token\") pod \"74cc2123-669b-4443-ba18-e694d315477f\" (UID: \"74cc2123-669b-4443-ba18-e694d315477f\") " Jul 30 18:12:35.023653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023571 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:12:35.023653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023621 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74cc2123-669b-4443-ba18-e694d315477f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:12:35.024110 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023838 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.024110 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023861 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.024110 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023876 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74cc2123-669b-4443-ba18-e694d315477f-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.024110 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.023890 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.024490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.024464 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-data" (OuterVolumeSpecName: "istio-data") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:12:35.025598 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.025570 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-kube-api-access-49qg7" (OuterVolumeSpecName: "kube-api-access-49qg7") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "kube-api-access-49qg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:12:35.025711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.025668 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:12:35.025711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.025698 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74cc2123-669b-4443-ba18-e694d315477f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:12:35.026070 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.026045 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-istio-token" (OuterVolumeSpecName: "istio-token") pod "74cc2123-669b-4443-ba18-e694d315477f" (UID: "74cc2123-669b-4443-ba18-e694d315477f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:12:35.124993 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.124970 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49qg7\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-kube-api-access-49qg7\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.124993 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.124991 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.125156 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.125000 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74cc2123-669b-4443-ba18-e694d315477f-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.125156 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.125010 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74cc2123-669b-4443-ba18-e694d315477f-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.125156 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.125022 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74cc2123-669b-4443-ba18-e694d315477f-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:12:35.886258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.886227 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" event={"ID":"74cc2123-669b-4443-ba18-e694d315477f","Type":"ContainerDied","Data":"1883e2fdd5d746a1b097176a7af5eb038106aa455e609706fdd429476ac27e60"} Jul 30 18:12:35.886258 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.886257 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4" Jul 30 18:12:35.886718 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.886283 2550 scope.go:117] "RemoveContainer" containerID="fbcb4298574bf926db0d3fd4c37ec5cd34cefba0e31a34061c98adb199240ec7" Jul 30 18:12:35.906267 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.906229 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4"] Jul 30 18:12:35.909348 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:35.909326 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-657b86df-openshift-default-798d58545b-7gnc4"] Jul 30 18:12:37.807960 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:12:37.807929 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74cc2123-669b-4443-ba18-e694d315477f" path="/var/lib/kubelet/pods/74cc2123-669b-4443-ba18-e694d315477f/volumes" Jul 30 18:13:45.565266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565229 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m"] Jul 30 18:13:45.565721 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565631 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" Jul 30 18:13:45.565721 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565646 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f719f2-72e4-44a2-83ba-0a3f20da12aa" containerName="istio-proxy" Jul 30 18:13:45.565721 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565655 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" Jul 30 18:13:45.565721 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565660 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" Jul 30 18:13:45.565721 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565677 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" Jul 30 18:13:45.565721 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565682 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" Jul 30 18:13:45.565932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565744 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" Jul 30 18:13:45.565932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.565756 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="74cc2123-669b-4443-ba18-e694d315477f" containerName="istio-proxy" Jul 30 18:13:45.568609 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.568595 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.571267 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.571244 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-166ba9-openshift-default-dockercfg-nlwhr\"" Jul 30 18:13:45.581512 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.581490 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m"] Jul 30 18:13:45.670157 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670126 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ec0365-c19a-49b3-8200-490bcccfef58-istiod-ca-cert\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670350 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670167 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-socket\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670350 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670245 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-token\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670350 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670321 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-podinfo\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670514 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670351 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-credential-socket\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670514 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670386 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-data\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670514 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670464 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-certs\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670514 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670495 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-envoy\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.670714 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.670547 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98pcx\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-kube-api-access-98pcx\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.771801 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.771766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-token\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.771846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-podinfo\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.771874 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-credential-socket\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.771904 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-data\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.771947 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-certs\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.771967 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-envoy\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772008 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98pcx\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-kube-api-access-98pcx\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772344 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772038 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ec0365-c19a-49b3-8200-490bcccfef58-istiod-ca-cert\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772415 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772358 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-data\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772415 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772328 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-credential-socket\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772415 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772393 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-socket\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772618 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772599 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-certs\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772678 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772616 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-socket\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.772903 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.772849 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ec0365-c19a-49b3-8200-490bcccfef58-istiod-ca-cert\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.774353 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.774333 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-envoy\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.774597 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.774579 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-podinfo\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.780049 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.780029 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-token\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.780260 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.780243 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98pcx\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-kube-api-access-98pcx\") pod \"e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:45.880950 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.880926 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-166ba9-openshift-default-dockercfg-nlwhr\"" Jul 30 18:13:45.888868 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:45.888851 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:46.030715 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.030690 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m"] Jul 30 18:13:46.032753 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:13:46.032723 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ec0365_c19a_49b3_8200_490bcccfef58.slice/crio-9a36cb72f0ceb7fa41e33239270f6d3a00a900d25d1233353d2c1a41472d5d00 WatchSource:0}: Error finding container 9a36cb72f0ceb7fa41e33239270f6d3a00a900d25d1233353d2c1a41472d5d00: Status 404 returned error can't find the container with id 9a36cb72f0ceb7fa41e33239270f6d3a00a900d25d1233353d2c1a41472d5d00 Jul 30 18:13:46.034742 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.034712 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:13:46.034871 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.034790 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:13:46.034871 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.034836 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:13:46.245239 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.245147 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" event={"ID":"e0ec0365-c19a-49b3-8200-490bcccfef58","Type":"ContainerStarted","Data":"5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8"} Jul 30 18:13:46.245239 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.245214 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" event={"ID":"e0ec0365-c19a-49b3-8200-490bcccfef58","Type":"ContainerStarted","Data":"9a36cb72f0ceb7fa41e33239270f6d3a00a900d25d1233353d2c1a41472d5d00"} Jul 30 18:13:46.272847 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.272765 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podStartSLOduration=1.272749791 podStartE2EDuration="1.272749791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:13:46.269541017 +0000 UTC m=+2821.002055698" watchObservedRunningTime="2026-07-30 18:13:46.272749791 +0000 UTC m=+2821.005264473" Jul 30 18:13:46.889178 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:46.889150 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:47.894012 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:47.893984 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:48.254920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:48.254840 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:13:48.255862 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:13:48.255801 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:14:02.324728 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:02.324702 2550 generic.go:356] "Generic (PLEG): container finished" podID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerID="5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8" exitCode=0 Jul 30 18:14:02.325172 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:02.324776 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" event={"ID":"e0ec0365-c19a-49b3-8200-490bcccfef58","Type":"ContainerDied","Data":"5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8"} Jul 30 18:14:02.325260 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:02.325244 2550 scope.go:117] "RemoveContainer" containerID="5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8" Jul 30 18:14:02.327194 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:02.326640 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:02.327194 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:02.326805 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:02.327194 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:02.326927 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:03.254914 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:03.254866 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:14:03.335170 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:03.335133 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" event={"ID":"e0ec0365-c19a-49b3-8200-490bcccfef58","Type":"ContainerStarted","Data":"a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923"} Jul 30 18:14:03.889900 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:03.889858 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:14:03.890801 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:03.890781 2550 patch_prober.go:28] interesting pod/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 30 18:14:03.890927 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:03.890845 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 30 18:14:04.889582 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:04.889546 2550 patch_prober.go:28] interesting pod/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 30 18:14:04.890015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:04.889625 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 30 18:14:05.890004 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:05.889970 2550 patch_prober.go:28] interesting pod/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 30 18:14:05.890399 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:05.890021 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 30 18:14:06.890072 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:06.890036 2550 patch_prober.go:28] interesting pod/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 30 18:14:06.890481 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:06.890087 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 30 18:14:07.893012 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:07.892979 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:14:07.893407 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:07.893242 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:14:07.893783 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:07.893763 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:14:14.011985 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.011946 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq"] Jul 30 18:14:14.015038 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.015012 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.017607 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.017584 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-984898-openshift-default-dockercfg-74ckg\"" Jul 30 18:14:14.027377 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.027355 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq"] Jul 30 18:14:14.134218 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134186 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-credential-socket\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134218 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134226 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-podinfo\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134466 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134372 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-socket\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134466 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-envoy\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134466 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134449 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-token\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134636 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134497 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-data\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134636 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134528 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkvh6\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-kube-api-access-bkvh6\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134636 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134562 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-certs\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.134636 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.134590 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1c7729-a220-4f57-9f71-0eb3d073d649-istiod-ca-cert\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235293 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235265 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-data\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235293 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235296 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkvh6\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-kube-api-access-bkvh6\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235497 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235323 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-certs\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235497 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235413 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1c7729-a220-4f57-9f71-0eb3d073d649-istiod-ca-cert\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235508 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-credential-socket\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235536 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-podinfo\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235614 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-socket\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235640 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-envoy\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235667 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-token\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235682 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-data\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.235711 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.235703 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-certs\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.236033 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.236003 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-socket\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.236089 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.236029 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-credential-socket\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.236205 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.236188 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1c7729-a220-4f57-9f71-0eb3d073d649-istiod-ca-cert\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.238049 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.238029 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-envoy\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.238303 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.238283 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-podinfo\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.243416 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.243391 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-token\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.243691 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.243672 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkvh6\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-kube-api-access-bkvh6\") pod \"e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.325585 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.325526 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:14.462147 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.462122 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq"] Jul 30 18:14:14.463423 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:14:14.463395 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1c7729_a220_4f57_9f71_0eb3d073d649.slice/crio-3f61022227d48f65af779588a56518c76d45cfecbfb30ec89b556059592b6aa9 WatchSource:0}: Error finding container 3f61022227d48f65af779588a56518c76d45cfecbfb30ec89b556059592b6aa9: Status 404 returned error can't find the container with id 3f61022227d48f65af779588a56518c76d45cfecbfb30ec89b556059592b6aa9 Jul 30 18:14:14.465493 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.465466 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:14.465565 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.465526 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:14.465565 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:14.465555 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:15.400832 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:15.400774 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" event={"ID":"0e1c7729-a220-4f57-9f71-0eb3d073d649","Type":"ContainerStarted","Data":"4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d"} Jul 30 18:14:15.401291 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:15.400857 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" event={"ID":"0e1c7729-a220-4f57-9f71-0eb3d073d649","Type":"ContainerStarted","Data":"3f61022227d48f65af779588a56518c76d45cfecbfb30ec89b556059592b6aa9"} Jul 30 18:14:15.425853 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:15.425790 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" podStartSLOduration=2.425775051 podStartE2EDuration="2.425775051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:14:15.422760783 +0000 UTC m=+2850.155275467" watchObservedRunningTime="2026-07-30 18:14:15.425775051 +0000 UTC m=+2850.158289733" Jul 30 18:14:16.325857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:16.325806 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:16.330541 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:16.330517 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:16.406655 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:16.406629 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:16.407501 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:16.407485 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:30.482469 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:30.482437 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerID="4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d" exitCode=0 Jul 30 18:14:30.482926 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:30.482510 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" event={"ID":"0e1c7729-a220-4f57-9f71-0eb3d073d649","Type":"ContainerDied","Data":"4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d"} Jul 30 18:14:30.482926 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:30.482866 2550 scope.go:117] "RemoveContainer" containerID="4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d" Jul 30 18:14:30.483862 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:30.483834 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:30.483972 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:30.483916 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:30.483972 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:30.483962 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:14:31.406871 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:31.406829 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:31.489309 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:31.489269 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" event={"ID":"0e1c7729-a220-4f57-9f71-0eb3d073d649","Type":"ContainerStarted","Data":"d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f"} Jul 30 18:14:32.328038 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:32.328001 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:32.328602 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:32.328568 2550 patch_prober.go:28] interesting pod/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 18:14:32.328713 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:32.328652 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 18:14:33.325974 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:33.325935 2550 patch_prober.go:28] interesting pod/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 18:14:33.326443 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:33.326029 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 18:14:34.326318 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:34.326280 2550 patch_prober.go:28] interesting pod/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 18:14:34.326947 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:34.326906 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 18:14:35.327000 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:35.326960 2550 patch_prober.go:28] interesting pod/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 18:14:35.327625 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:35.327584 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 18:14:36.330337 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:36.330308 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:36.330768 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:36.330593 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:14:36.331340 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:14:36.331320 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:16:03.731348 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.731312 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47"] Jul 30 18:16:03.733772 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.733754 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:03.736545 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.736523 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 18:16:03.736680 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.736559 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qw74k\"" Jul 30 18:16:03.743087 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.743061 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47"] Jul 30 18:16:03.793594 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.793568 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba042a83-b894-45ab-ab7c-1f7d6054c67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47\" (UID: \"ba042a83-b894-45ab-ab7c-1f7d6054c67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:03.894277 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.894247 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba042a83-b894-45ab-ab7c-1f7d6054c67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47\" (UID: \"ba042a83-b894-45ab-ab7c-1f7d6054c67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:03.894834 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:03.894795 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba042a83-b894-45ab-ab7c-1f7d6054c67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47\" (UID: \"ba042a83-b894-45ab-ab7c-1f7d6054c67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:04.045050 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:04.044966 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:04.177960 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:04.177935 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47"] Jul 30 18:16:04.179907 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:16:04.179884 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba042a83_b894_45ab_ab7c_1f7d6054c67d.slice/crio-0db025d2f46abed1eca3b9617828c0b918ea929dabb28b946520f97f746de6dd WatchSource:0}: Error finding container 0db025d2f46abed1eca3b9617828c0b918ea929dabb28b946520f97f746de6dd: Status 404 returned error can't find the container with id 0db025d2f46abed1eca3b9617828c0b918ea929dabb28b946520f97f746de6dd Jul 30 18:16:04.182240 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:04.182225 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 18:16:04.964341 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:04.964309 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" event={"ID":"ba042a83-b894-45ab-ab7c-1f7d6054c67d","Type":"ContainerStarted","Data":"0db025d2f46abed1eca3b9617828c0b918ea929dabb28b946520f97f746de6dd"} Jul 30 18:16:07.984146 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:07.984108 2550 generic.go:356] "Generic (PLEG): container finished" podID="ba042a83-b894-45ab-ab7c-1f7d6054c67d" containerID="7f825188ea3ee9e1da8c8d12d7b7ac74beea494ff0e8cdae59826f46dfaaa97f" exitCode=0 Jul 30 18:16:07.984516 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:07.984193 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" event={"ID":"ba042a83-b894-45ab-ab7c-1f7d6054c67d","Type":"ContainerDied","Data":"7f825188ea3ee9e1da8c8d12d7b7ac74beea494ff0e8cdae59826f46dfaaa97f"} Jul 30 18:16:09.121283 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.121259 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:09.240733 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.240657 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba042a83-b894-45ab-ab7c-1f7d6054c67d-maas-api-service-ca\") pod \"ba042a83-b894-45ab-ab7c-1f7d6054c67d\" (UID: \"ba042a83-b894-45ab-ab7c-1f7d6054c67d\") " Jul 30 18:16:09.241005 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.240981 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba042a83-b894-45ab-ab7c-1f7d6054c67d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba042a83-b894-45ab-ab7c-1f7d6054c67d" (UID: "ba042a83-b894-45ab-ab7c-1f7d6054c67d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:16:09.341603 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.341575 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba042a83-b894-45ab-ab7c-1f7d6054c67d-maas-api-service-ca\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:09.994986 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.994959 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" Jul 30 18:16:09.995144 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.995000 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47" event={"ID":"ba042a83-b894-45ab-ab7c-1f7d6054c67d","Type":"ContainerDied","Data":"0db025d2f46abed1eca3b9617828c0b918ea929dabb28b946520f97f746de6dd"} Jul 30 18:16:09.995144 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:09.995044 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0db025d2f46abed1eca3b9617828c0b918ea929dabb28b946520f97f746de6dd" Jul 30 18:16:14.395745 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:14.395707 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47"] Jul 30 18:16:14.400350 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:14.400321 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-166ba9-jdk47"] Jul 30 18:16:15.808335 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:15.808304 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba042a83-b894-45ab-ab7c-1f7d6054c67d" path="/var/lib/kubelet/pods/ba042a83-b894-45ab-ab7c-1f7d6054c67d/volumes" Jul 30 18:16:21.258462 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:21.258431 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m"] Jul 30 18:16:21.258912 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:21.258678 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" containerID="cri-o://a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923" gracePeriod=30 Jul 30 18:16:22.895872 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:22.895780 2550 patch_prober.go:28] interesting pod/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 18:16:22.896215 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:22.895871 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 18:16:26.513125 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.513103 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:16:26.696406 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696378 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-socket\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696543 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696416 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-podinfo\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696543 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696446 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-envoy\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696616 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696552 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-credential-socket\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696616 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696586 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-certs\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696623 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98pcx\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-kube-api-access-98pcx\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696675 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:26.696722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696682 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ec0365-c19a-49b3-8200-490bcccfef58-istiod-ca-cert\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696726 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-token\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696785 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-data\") pod \"e0ec0365-c19a-49b3-8200-490bcccfef58\" (UID: \"e0ec0365-c19a-49b3-8200-490bcccfef58\") " Jul 30 18:16:26.696920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.696897 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:26.697092 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.697004 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0ec0365-c19a-49b3-8200-490bcccfef58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:16:26.697227 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.697209 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.697312 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.697232 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ec0365-c19a-49b3-8200-490bcccfef58-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.697312 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.697245 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.697394 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.697235 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:26.697914 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.697890 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:26.698835 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.698797 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:16:26.698938 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.698797 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:26.699075 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.699052 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-kube-api-access-98pcx" (OuterVolumeSpecName: "kube-api-access-98pcx") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "kube-api-access-98pcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:16:26.699275 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.699256 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0ec0365-c19a-49b3-8200-490bcccfef58" (UID: "e0ec0365-c19a-49b3-8200-490bcccfef58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:16:26.798418 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.798392 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98pcx\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-kube-api-access-98pcx\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.798418 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.798415 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.798541 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.798423 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.798541 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.798431 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.798541 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.798439 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:26.798541 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:26.798447 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ec0365-c19a-49b3-8200-490bcccfef58-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:27.084168 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.084074 2550 generic.go:356] "Generic (PLEG): container finished" podID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerID="a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923" exitCode=0 Jul 30 18:16:27.084168 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.084121 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" Jul 30 18:16:27.084404 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.084167 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" event={"ID":"e0ec0365-c19a-49b3-8200-490bcccfef58","Type":"ContainerDied","Data":"a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923"} Jul 30 18:16:27.084404 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.084228 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m" event={"ID":"e0ec0365-c19a-49b3-8200-490bcccfef58","Type":"ContainerDied","Data":"9a36cb72f0ceb7fa41e33239270f6d3a00a900d25d1233353d2c1a41472d5d00"} Jul 30 18:16:27.084404 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.084259 2550 scope.go:117] "RemoveContainer" containerID="a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923" Jul 30 18:16:27.094838 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.094799 2550 scope.go:117] "RemoveContainer" containerID="5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8" Jul 30 18:16:27.103289 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.103273 2550 scope.go:117] "RemoveContainer" containerID="a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923" Jul 30 18:16:27.103506 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:16:27.103488 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923\": container with ID starting with a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923 not found: ID does not exist" containerID="a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923" Jul 30 18:16:27.103568 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.103512 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923"} err="failed to get container status \"a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923\": rpc error: code = NotFound desc = could not find container \"a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923\": container with ID starting with a44408879159277e3880285bbaec7f52270ede99f64217636be7e93a258b4923 not found: ID does not exist" Jul 30 18:16:27.103568 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.103529 2550 scope.go:117] "RemoveContainer" containerID="5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8" Jul 30 18:16:27.103894 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:16:27.103831 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8\": container with ID starting with 5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8 not found: ID does not exist" containerID="5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8" Jul 30 18:16:27.103987 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.103896 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8"} err="failed to get container status \"5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8\": rpc error: code = NotFound desc = could not find container \"5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8\": container with ID starting with 5b15de9019e43b3054db0b120d452a553315380f282db55b6b7c1666c6cac8a8 not found: ID does not exist" Jul 30 18:16:27.106921 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.106901 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m"] Jul 30 18:16:27.110291 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.110269 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-166ba9-openshift-default-74db895888-k6j9m"] Jul 30 18:16:27.808404 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:27.808362 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" path="/var/lib/kubelet/pods/e0ec0365-c19a-49b3-8200-490bcccfef58/volumes" Jul 30 18:16:39.357707 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:39.357674 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq"] Jul 30 18:16:39.358200 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:39.357942 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" containerID="cri-o://d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f" gracePeriod=30 Jul 30 18:16:40.281147 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281109 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x"] Jul 30 18:16:40.281617 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281603 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" Jul 30 18:16:40.281667 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281621 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" Jul 30 18:16:40.281667 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281633 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba042a83-b894-45ab-ab7c-1f7d6054c67d" containerName="revoke-keys" Jul 30 18:16:40.281667 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281641 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba042a83-b894-45ab-ab7c-1f7d6054c67d" containerName="revoke-keys" Jul 30 18:16:40.281762 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281666 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" Jul 30 18:16:40.281762 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281675 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" Jul 30 18:16:40.281762 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281753 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" Jul 30 18:16:40.281884 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281763 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0ec0365-c19a-49b3-8200-490bcccfef58" containerName="istio-proxy" Jul 30 18:16:40.281884 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.281774 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba042a83-b894-45ab-ab7c-1f7d6054c67d" containerName="revoke-keys" Jul 30 18:16:40.290025 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.290003 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.292930 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.292906 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d12cf5-openshift-default-dockercfg-rj4vt\"" Jul 30 18:16:40.298324 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.298302 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x"] Jul 30 18:16:40.304463 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304438 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istiod-ca-cert\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304570 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304488 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-podinfo\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304570 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304552 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-envoy\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304644 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304604 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-token\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304679 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304642 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-socket\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304679 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304665 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d6c4\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-kube-api-access-9d6c4\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304759 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304694 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-data\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304858 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304838 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-credential-socket\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.304923 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.304909 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-certs\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.405862 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.405803 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-socket\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406066 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d6c4\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-kube-api-access-9d6c4\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406108 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-data\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406193 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-credential-socket\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406254 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-certs\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406290 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istiod-ca-cert\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406363 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406324 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-podinfo\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406672 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406365 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-envoy\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406672 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406408 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-token\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406672 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406457 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-socket\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406672 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406468 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-data\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.406915 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.406712 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-credential-socket\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.407149 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.407125 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istiod-ca-cert\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.407261 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.407205 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-certs\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.409531 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.409510 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-envoy\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.409618 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.409597 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-podinfo\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.414254 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.414232 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-token\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.414796 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.414777 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d6c4\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-kube-api-access-9d6c4\") pod \"e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.603023 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.602998 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:40.741623 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.741598 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x"] Jul 30 18:16:40.743432 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:16:40.743404 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1041b4ab_bf9f_45bb_82fa_616b32ba3003.slice/crio-dbded468125f5987d58c25941d278edfafe24f52f2e1022a1ebfa7f85459f142 WatchSource:0}: Error finding container dbded468125f5987d58c25941d278edfafe24f52f2e1022a1ebfa7f85459f142: Status 404 returned error can't find the container with id dbded468125f5987d58c25941d278edfafe24f52f2e1022a1ebfa7f85459f142 Jul 30 18:16:40.745700 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.745665 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:16:40.745792 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.745750 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:16:40.745864 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:40.745800 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:16:41.167951 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:41.167901 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" event={"ID":"1041b4ab-bf9f-45bb-82fa-616b32ba3003","Type":"ContainerStarted","Data":"8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1"} Jul 30 18:16:41.168126 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:41.168005 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" event={"ID":"1041b4ab-bf9f-45bb-82fa-616b32ba3003","Type":"ContainerStarted","Data":"dbded468125f5987d58c25941d278edfafe24f52f2e1022a1ebfa7f85459f142"} Jul 30 18:16:41.189625 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:41.189583 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podStartSLOduration=1.189569473 podStartE2EDuration="1.189569473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:16:41.187912175 +0000 UTC m=+2995.920426858" watchObservedRunningTime="2026-07-30 18:16:41.189569473 +0000 UTC m=+2995.922084157" Jul 30 18:16:41.603689 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:41.603661 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:41.608234 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:41.608213 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:42.173435 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:42.173405 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:42.174368 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:42.174347 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:16:44.613124 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.613101 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:16:44.644118 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644061 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-token\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644263 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644129 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-credential-socket\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644263 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644185 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-podinfo\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644263 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644210 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-socket\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644263 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644234 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-envoy\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644266 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1c7729-a220-4f57-9f71-0eb3d073d649-istiod-ca-cert\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644292 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-data\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644326 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-certs\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644378 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkvh6\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-kube-api-access-bkvh6\") pod \"0e1c7729-a220-4f57-9f71-0eb3d073d649\" (UID: \"0e1c7729-a220-4f57-9f71-0eb3d073d649\") " Jul 30 18:16:44.644696 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644545 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:44.644778 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644754 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.644869 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644793 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:44.645014 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.644980 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:44.645275 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.645228 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1c7729-a220-4f57-9f71-0eb3d073d649-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:16:44.645381 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.645359 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:44.647015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.646985 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:16:44.647684 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.647624 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:16:44.647853 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.647743 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:16:44.648064 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.648044 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-kube-api-access-bkvh6" (OuterVolumeSpecName: "kube-api-access-bkvh6") pod "0e1c7729-a220-4f57-9f71-0eb3d073d649" (UID: "0e1c7729-a220-4f57-9f71-0eb3d073d649"). InnerVolumeSpecName "kube-api-access-bkvh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:16:44.745456 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745426 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745456 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745449 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745456 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745459 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745716 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745467 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1c7729-a220-4f57-9f71-0eb3d073d649-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745716 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745475 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745716 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745483 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1c7729-a220-4f57-9f71-0eb3d073d649-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745716 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745491 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkvh6\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-kube-api-access-bkvh6\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:44.745716 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:44.745500 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1c7729-a220-4f57-9f71-0eb3d073d649-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:16:45.190855 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.190805 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerID="d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f" exitCode=0 Jul 30 18:16:45.191015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.190875 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" Jul 30 18:16:45.191015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.190880 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" event={"ID":"0e1c7729-a220-4f57-9f71-0eb3d073d649","Type":"ContainerDied","Data":"d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f"} Jul 30 18:16:45.191015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.190929 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq" event={"ID":"0e1c7729-a220-4f57-9f71-0eb3d073d649","Type":"ContainerDied","Data":"3f61022227d48f65af779588a56518c76d45cfecbfb30ec89b556059592b6aa9"} Jul 30 18:16:45.191015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.190952 2550 scope.go:117] "RemoveContainer" containerID="d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f" Jul 30 18:16:45.201691 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.201676 2550 scope.go:117] "RemoveContainer" containerID="4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d" Jul 30 18:16:45.209582 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.209564 2550 scope.go:117] "RemoveContainer" containerID="d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f" Jul 30 18:16:45.209848 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:16:45.209830 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f\": container with ID starting with d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f not found: ID does not exist" containerID="d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f" Jul 30 18:16:45.209905 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.209856 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f"} err="failed to get container status \"d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f\": rpc error: code = NotFound desc = could not find container \"d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f\": container with ID starting with d12767fe45c78d531e80fe824030e57e12907f93a7fe68ed9231d4a40c06d45f not found: ID does not exist" Jul 30 18:16:45.209905 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.209871 2550 scope.go:117] "RemoveContainer" containerID="4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d" Jul 30 18:16:45.210115 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:16:45.210101 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d\": container with ID starting with 4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d not found: ID does not exist" containerID="4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d" Jul 30 18:16:45.210160 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.210120 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d"} err="failed to get container status \"4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d\": rpc error: code = NotFound desc = could not find container \"4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d\": container with ID starting with 4876a2c8650502292f942d2ed1228ae6cde8d63f268c3b347835dda44083c52d not found: ID does not exist" Jul 30 18:16:45.214265 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.214244 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq"] Jul 30 18:16:45.218474 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.218451 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-984898-openshift-default-76dbcb5688-hjqcq"] Jul 30 18:16:45.813421 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:45.813388 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" path="/var/lib/kubelet/pods/0e1c7729-a220-4f57-9f71-0eb3d073d649/volumes" Jul 30 18:16:58.214988 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:58.214942 2550 patch_prober.go:28] interesting pod/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 18:16:58.215380 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:16:58.215020 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 18:17:00.276670 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:00.276637 2550 generic.go:356] "Generic (PLEG): container finished" podID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerID="8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1" exitCode=0 Jul 30 18:17:00.277135 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:00.276709 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" event={"ID":"1041b4ab-bf9f-45bb-82fa-616b32ba3003","Type":"ContainerDied","Data":"8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1"} Jul 30 18:17:00.277135 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:00.277109 2550 scope.go:117] "RemoveContainer" containerID="8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1" Jul 30 18:17:00.278062 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:00.278037 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:00.278162 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:00.278105 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:00.278162 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:00.278138 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:01.284431 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:01.284378 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" event={"ID":"1041b4ab-bf9f-45bb-82fa-616b32ba3003","Type":"ContainerStarted","Data":"81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5"} Jul 30 18:17:01.603473 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:01.603442 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:17:01.603649 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:01.603628 2550 patch_prober.go:28] interesting pod/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 18:17:01.603691 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:01.603674 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 18:17:02.603381 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:02.603349 2550 patch_prober.go:28] interesting pod/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 18:17:02.603767 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:02.603400 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 18:17:03.603605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:03.603571 2550 patch_prober.go:28] interesting pod/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 18:17:03.604019 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:03.603628 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 18:17:04.603429 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:04.603390 2550 patch_prober.go:28] interesting pod/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 18:17:04.603621 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:04.603445 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 18:17:05.606499 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:05.606468 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:17:05.606876 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:05.606717 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:17:05.607254 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:05.607239 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:17:09.113298 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.113266 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6"] Jul 30 18:17:09.113768 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.113620 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" Jul 30 18:17:09.113768 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.113632 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" Jul 30 18:17:09.113768 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.113712 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" Jul 30 18:17:09.115849 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.115832 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.119203 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.119185 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b51ede-openshift-default-dockercfg-qkzq9\"" Jul 30 18:17:09.129009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.128989 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6"] Jul 30 18:17:09.256522 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256487 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-credential-socket\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256724 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256526 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-socket\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256724 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256562 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-certs\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256724 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256583 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-envoy\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256724 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256667 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-istio-token\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256958 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256732 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d571fd4-6304-48be-8076-427216ccf88d-istio-podinfo\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256958 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256846 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d571fd4-6304-48be-8076-427216ccf88d-istiod-ca-cert\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256958 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256894 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw5d8\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-kube-api-access-gw5d8\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.256958 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.256925 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-data\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.357994 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.357967 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d571fd4-6304-48be-8076-427216ccf88d-istiod-ca-cert\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358008 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw5d8\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-kube-api-access-gw5d8\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358034 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-data\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358050 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-credential-socket\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358075 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-socket\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358099 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-certs\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358114 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-envoy\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358150 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-istio-token\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358612 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358249 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d571fd4-6304-48be-8076-427216ccf88d-istio-podinfo\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358612 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358522 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-data\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358612 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358572 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-socket\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358612 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358599 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-certs\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358798 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358702 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-credential-socket\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.358798 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.358776 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d571fd4-6304-48be-8076-427216ccf88d-istiod-ca-cert\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.360537 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.360519 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-envoy\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.360649 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.360633 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d571fd4-6304-48be-8076-427216ccf88d-istio-podinfo\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.366041 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.365989 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-istio-token\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.366141 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.366065 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw5d8\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-kube-api-access-gw5d8\") pod \"e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.428202 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.428182 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:09.566428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.566396 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6"] Jul 30 18:17:09.567799 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:17:09.567772 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d571fd4_6304_48be_8076_427216ccf88d.slice/crio-58da4b77e639d07cb762b3268c52ebd1a33a9b14a69fda47c976e3d66735ed71 WatchSource:0}: Error finding container 58da4b77e639d07cb762b3268c52ebd1a33a9b14a69fda47c976e3d66735ed71: Status 404 returned error can't find the container with id 58da4b77e639d07cb762b3268c52ebd1a33a9b14a69fda47c976e3d66735ed71 Jul 30 18:17:09.569975 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.569932 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:09.570059 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.570025 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:09.570098 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:09.570067 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:10.334330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:10.334293 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" event={"ID":"9d571fd4-6304-48be-8076-427216ccf88d","Type":"ContainerStarted","Data":"4ad22071e4f91e951c0a43015825ecc2ff2dad2195fbe7b4c8825e077750c559"} Jul 30 18:17:10.334738 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:10.334346 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" event={"ID":"9d571fd4-6304-48be-8076-427216ccf88d","Type":"ContainerStarted","Data":"58da4b77e639d07cb762b3268c52ebd1a33a9b14a69fda47c976e3d66735ed71"} Jul 30 18:17:10.356316 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:10.356261 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" podStartSLOduration=1.35624682 podStartE2EDuration="1.35624682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:17:10.354229838 +0000 UTC m=+3025.086744521" watchObservedRunningTime="2026-07-30 18:17:10.35624682 +0000 UTC m=+3025.088761502" Jul 30 18:17:10.428901 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:10.428877 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:10.434117 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:10.434092 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:11.339897 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:11.339868 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:11.340873 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:11.340850 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:14.144609 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.144579 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz"] Jul 30 18:17:14.145198 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.145156 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" Jul 30 18:17:14.145198 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.145177 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" Jul 30 18:17:14.145330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.145290 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1c7729-a220-4f57-9f71-0eb3d073d649" containerName="istio-proxy" Jul 30 18:17:14.147310 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.147289 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.150128 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.150107 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b51ede\"" Jul 30 18:17:14.150128 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.150122 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b51ede-dockercfg-mbmzw\"" Jul 30 18:17:14.160022 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.159996 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz"] Jul 30 18:17:14.304943 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.304909 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frngr\" (UniqueName: \"kubernetes.io/projected/f5956276-9660-43cc-8524-0345551698f0-kube-api-access-frngr\") pod \"payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.305129 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.304968 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5956276-9660-43cc-8524-0345551698f0-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.406073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.405995 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5956276-9660-43cc-8524-0345551698f0-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.406230 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.406118 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frngr\" (UniqueName: \"kubernetes.io/projected/f5956276-9660-43cc-8524-0345551698f0-kube-api-access-frngr\") pod \"payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.406650 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.406632 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5956276-9660-43cc-8524-0345551698f0-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.414798 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.414767 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frngr\" (UniqueName: \"kubernetes.io/projected/f5956276-9660-43cc-8524-0345551698f0-kube-api-access-frngr\") pod \"payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.459621 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.459597 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:14.609800 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:14.609775 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz"] Jul 30 18:17:14.611356 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:17:14.611331 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5956276_9660_43cc_8524_0345551698f0.slice/crio-f352882dd36d44a721402196b30e5e8d033b1b942aa860e46772396b526675a4 WatchSource:0}: Error finding container f352882dd36d44a721402196b30e5e8d033b1b942aa860e46772396b526675a4: Status 404 returned error can't find the container with id f352882dd36d44a721402196b30e5e8d033b1b942aa860e46772396b526675a4 Jul 30 18:17:15.367068 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:15.367036 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" event={"ID":"f5956276-9660-43cc-8524-0345551698f0","Type":"ContainerStarted","Data":"d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537"} Jul 30 18:17:15.367499 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:15.367082 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" event={"ID":"f5956276-9660-43cc-8524-0345551698f0","Type":"ContainerStarted","Data":"f352882dd36d44a721402196b30e5e8d033b1b942aa860e46772396b526675a4"} Jul 30 18:17:15.367499 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:15.367178 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:15.383344 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:15.383302 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" podStartSLOduration=1.383289569 podStartE2EDuration="1.383289569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:17:15.381725872 +0000 UTC m=+3030.114240558" watchObservedRunningTime="2026-07-30 18:17:15.383289569 +0000 UTC m=+3030.115804252" Jul 30 18:17:26.375112 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:26.375075 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:17:27.355240 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.355118 2550 patch_prober.go:28] interesting pod/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 18:17:27.355352 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.355230 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 18:17:27.440609 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.440580 2550 generic.go:356] "Generic (PLEG): container finished" podID="9d571fd4-6304-48be-8076-427216ccf88d" containerID="4ad22071e4f91e951c0a43015825ecc2ff2dad2195fbe7b4c8825e077750c559" exitCode=0 Jul 30 18:17:27.441016 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.440658 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" event={"ID":"9d571fd4-6304-48be-8076-427216ccf88d","Type":"ContainerDied","Data":"4ad22071e4f91e951c0a43015825ecc2ff2dad2195fbe7b4c8825e077750c559"} Jul 30 18:17:27.441177 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.441161 2550 scope.go:117] "RemoveContainer" containerID="4ad22071e4f91e951c0a43015825ecc2ff2dad2195fbe7b4c8825e077750c559" Jul 30 18:17:27.442242 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.442218 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:27.442295 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.442282 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:27.442334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:27.442311 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:17:28.450308 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:28.450274 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" event={"ID":"9d571fd4-6304-48be-8076-427216ccf88d","Type":"ContainerStarted","Data":"8ab6967a5207cdbfc94aabde5423f16c74581fbd9e8a3c717bfa698d7f53743b"} Jul 30 18:17:29.429133 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:29.429074 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:29.429348 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:29.429218 2550 patch_prober.go:28] interesting pod/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 18:17:29.429348 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:29.429261 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 18:17:30.429005 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:30.428959 2550 patch_prober.go:28] interesting pod/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 18:17:30.429533 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:30.429029 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 18:17:31.428683 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:31.428648 2550 patch_prober.go:28] interesting pod/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 18:17:31.428997 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:31.428965 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 18:17:32.432144 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:32.432115 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:32.432656 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:32.432632 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:17:32.433398 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:17:32.433376 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:18:02.705343 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:02.705257 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x"] Jul 30 18:18:02.705921 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:02.705508 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" containerID="cri-o://81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5" gracePeriod=30 Jul 30 18:18:05.609216 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:05.609182 2550 patch_prober.go:28] interesting pod/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 18:18:05.609594 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:05.609252 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 18:18:07.949167 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:07.949141 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:18:08.060440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060343 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-certs\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060380 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-data\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060401 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-socket\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060433 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d6c4\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-kube-api-access-9d6c4\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060466 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-token\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060502 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-envoy\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060589 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-credential-socket\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060655 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:08.060789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060756 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istiod-ca-cert\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.060789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060790 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-podinfo\") pod \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\" (UID: \"1041b4ab-bf9f-45bb-82fa-616b32ba3003\") " Jul 30 18:18:08.061160 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.060881 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:08.061160 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.061036 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:08.061274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.061227 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.061274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.061247 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.061274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.061252 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:18:08.061274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.061261 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.061494 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.061476 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-data" (OuterVolumeSpecName: "istio-data") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:08.063223 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.063194 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-kube-api-access-9d6c4" (OuterVolumeSpecName: "kube-api-access-9d6c4") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "kube-api-access-9d6c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:18:08.063223 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.063214 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:18:08.063473 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.063449 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-token" (OuterVolumeSpecName: "istio-token") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:18:08.063473 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.063451 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1041b4ab-bf9f-45bb-82fa-616b32ba3003" (UID: "1041b4ab-bf9f-45bb-82fa-616b32ba3003"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:08.162619 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.162585 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.162619 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.162614 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.162619 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.162622 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.162877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.162630 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d6c4\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-kube-api-access-9d6c4\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.162877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.162639 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.162877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.162647 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1041b4ab-bf9f-45bb-82fa-616b32ba3003-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:08.664518 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.664480 2550 generic.go:356] "Generic (PLEG): container finished" podID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerID="81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5" exitCode=0 Jul 30 18:18:08.664686 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.664528 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" Jul 30 18:18:08.664686 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.664527 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" event={"ID":"1041b4ab-bf9f-45bb-82fa-616b32ba3003","Type":"ContainerDied","Data":"81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5"} Jul 30 18:18:08.664686 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.664590 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x" event={"ID":"1041b4ab-bf9f-45bb-82fa-616b32ba3003","Type":"ContainerDied","Data":"dbded468125f5987d58c25941d278edfafe24f52f2e1022a1ebfa7f85459f142"} Jul 30 18:18:08.664686 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.664620 2550 scope.go:117] "RemoveContainer" containerID="81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5" Jul 30 18:18:08.674172 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.674156 2550 scope.go:117] "RemoveContainer" containerID="8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1" Jul 30 18:18:08.682430 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.682413 2550 scope.go:117] "RemoveContainer" containerID="81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5" Jul 30 18:18:08.682673 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:18:08.682655 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5\": container with ID starting with 81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5 not found: ID does not exist" containerID="81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5" Jul 30 18:18:08.682752 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.682687 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5"} err="failed to get container status \"81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5\": rpc error: code = NotFound desc = could not find container \"81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5\": container with ID starting with 81725945d94ecb67d31d9faf33813ad99bd065552083e574d2b54495c46000e5 not found: ID does not exist" Jul 30 18:18:08.682752 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.682712 2550 scope.go:117] "RemoveContainer" containerID="8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1" Jul 30 18:18:08.682980 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:18:08.682962 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1\": container with ID starting with 8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1 not found: ID does not exist" containerID="8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1" Jul 30 18:18:08.683023 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.682986 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1"} err="failed to get container status \"8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1\": rpc error: code = NotFound desc = could not find container \"8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1\": container with ID starting with 8ab04df91fafc5cbe3fe20fb4d013660272d27c40bee256907ab42793ddf57a1 not found: ID does not exist" Jul 30 18:18:08.687577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.687556 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x"] Jul 30 18:18:08.690932 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:08.690912 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d12cf5-openshift-default-6c467c8946-xfw7x"] Jul 30 18:18:09.807483 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:09.807447 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" path="/var/lib/kubelet/pods/1041b4ab-bf9f-45bb-82fa-616b32ba3003/volumes" Jul 30 18:18:18.763244 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:18.763211 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz"] Jul 30 18:18:18.763739 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:18.763471 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" podUID="f5956276-9660-43cc-8524-0345551698f0" containerName="payload-processing" containerID="cri-o://d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537" gracePeriod=30 Jul 30 18:18:19.017877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.017800 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:18:19.159922 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.159890 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5956276-9660-43cc-8524-0345551698f0-plugins-config-volume\") pod \"f5956276-9660-43cc-8524-0345551698f0\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " Jul 30 18:18:19.160110 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.159964 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frngr\" (UniqueName: \"kubernetes.io/projected/f5956276-9660-43cc-8524-0345551698f0-kube-api-access-frngr\") pod \"f5956276-9660-43cc-8524-0345551698f0\" (UID: \"f5956276-9660-43cc-8524-0345551698f0\") " Jul 30 18:18:19.160254 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.160229 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5956276-9660-43cc-8524-0345551698f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5956276-9660-43cc-8524-0345551698f0" (UID: "f5956276-9660-43cc-8524-0345551698f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:18:19.162182 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.162145 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5956276-9660-43cc-8524-0345551698f0-kube-api-access-frngr" (OuterVolumeSpecName: "kube-api-access-frngr") pod "f5956276-9660-43cc-8524-0345551698f0" (UID: "f5956276-9660-43cc-8524-0345551698f0"). InnerVolumeSpecName "kube-api-access-frngr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:18:19.261544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.261513 2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5956276-9660-43cc-8524-0345551698f0-plugins-config-volume\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:19.261544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.261539 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frngr\" (UniqueName: \"kubernetes.io/projected/f5956276-9660-43cc-8524-0345551698f0-kube-api-access-frngr\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:19.725637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.725599 2550 generic.go:356] "Generic (PLEG): container finished" podID="f5956276-9660-43cc-8524-0345551698f0" containerID="d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537" exitCode=0 Jul 30 18:18:19.725836 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.725659 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" event={"ID":"f5956276-9660-43cc-8524-0345551698f0","Type":"ContainerDied","Data":"d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537"} Jul 30 18:18:19.725836 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.725665 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" Jul 30 18:18:19.725836 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.725710 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz" event={"ID":"f5956276-9660-43cc-8524-0345551698f0","Type":"ContainerDied","Data":"f352882dd36d44a721402196b30e5e8d033b1b942aa860e46772396b526675a4"} Jul 30 18:18:19.725836 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.725743 2550 scope.go:117] "RemoveContainer" containerID="d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537" Jul 30 18:18:19.734592 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.734570 2550 scope.go:117] "RemoveContainer" containerID="d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537" Jul 30 18:18:19.734864 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:18:19.734845 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537\": container with ID starting with d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537 not found: ID does not exist" containerID="d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537" Jul 30 18:18:19.734951 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.734878 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537"} err="failed to get container status \"d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537\": rpc error: code = NotFound desc = could not find container \"d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537\": container with ID starting with d2bff391e27ad9f412cc7f982a8bbfb1f071131ca2c7a6a30a9336ac7e955537 not found: ID does not exist" Jul 30 18:18:19.747445 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.747403 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz"] Jul 30 18:18:19.751411 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.751388 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b51ede-6b79c9865d-dfpqz"] Jul 30 18:18:19.808707 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:19.808679 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5956276-9660-43cc-8524-0345551698f0" path="/var/lib/kubelet/pods/f5956276-9660-43cc-8524-0345551698f0/volumes" Jul 30 18:18:25.586916 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:25.586878 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6"] Jul 30 18:18:25.587319 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:25.587136 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" containerID="cri-o://8ab6967a5207cdbfc94aabde5423f16c74581fbd9e8a3c717bfa698d7f53743b" gracePeriod=30 Jul 30 18:18:26.514612 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.514522 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l"] Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515458 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515483 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515502 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5956276-9660-43cc-8524-0345551698f0" containerName="payload-processing" Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515511 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5956276-9660-43cc-8524-0345551698f0" containerName="payload-processing" Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515601 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5956276-9660-43cc-8524-0345551698f0" containerName="payload-processing" Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515614 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" Jul 30 18:18:26.516444 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.515623 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" Jul 30 18:18:26.518962 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.518943 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.521727 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.521704 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8f586d-openshift-default-dockercfg-5fvcg\"" Jul 30 18:18:26.530114 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.530095 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l"] Jul 30 18:18:26.622330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622292 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-socket\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622346 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istiod-ca-cert\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622432 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-certs\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622459 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-data\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622494 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-envoy\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622599 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tks4s\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-kube-api-access-tks4s\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622644 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-token\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622690 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-podinfo\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.622879 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.622734 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-credential-socket\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723306 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723268 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tks4s\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-kube-api-access-tks4s\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723306 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723306 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-token\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723336 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-podinfo\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723375 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-credential-socket\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723402 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-socket\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723428 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istiod-ca-cert\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723481 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-certs\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723577 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723569 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-data\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723944 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723623 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-envoy\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723944 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723877 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-credential-socket\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.723944 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723915 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-socket\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.724093 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.723980 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-certs\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.724093 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.724046 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-data\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.724242 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.724224 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istiod-ca-cert\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.725984 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.725961 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-envoy\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.726126 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.726108 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-podinfo\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.731653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.731633 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-token\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.732848 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.732798 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tks4s\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-kube-api-access-tks4s\") pod \"e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.833591 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.833513 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:26.976428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.976388 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:26.976549 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.976524 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:26.976625 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.976608 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:26.978039 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:26.978014 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l"] Jul 30 18:18:27.770875 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:27.770839 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" event={"ID":"f8555d7d-14f9-481d-8ee3-c09b188b0e34","Type":"ContainerStarted","Data":"5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9"} Jul 30 18:18:27.771245 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:27.770892 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" event={"ID":"f8555d7d-14f9-481d-8ee3-c09b188b0e34","Type":"ContainerStarted","Data":"cd311c6939c68db5425316dec49708fd5528cad0f45db192190e42aeda1f3f32"} Jul 30 18:18:27.793605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:27.793558 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" podStartSLOduration=1.793544241 podStartE2EDuration="1.793544241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:18:27.791833031 +0000 UTC m=+3102.524347714" watchObservedRunningTime="2026-07-30 18:18:27.793544241 +0000 UTC m=+3102.526058925" Jul 30 18:18:27.834400 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:27.834368 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:28.839499 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:28.839471 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:29.782448 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:29.782411 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:29.783529 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:29.783508 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:30.790754 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.790640 2550 generic.go:356] "Generic (PLEG): container finished" podID="9d571fd4-6304-48be-8076-427216ccf88d" containerID="8ab6967a5207cdbfc94aabde5423f16c74581fbd9e8a3c717bfa698d7f53743b" exitCode=0 Jul 30 18:18:30.791201 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.790787 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" event={"ID":"9d571fd4-6304-48be-8076-427216ccf88d","Type":"ContainerDied","Data":"8ab6967a5207cdbfc94aabde5423f16c74581fbd9e8a3c717bfa698d7f53743b"} Jul 30 18:18:30.791201 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.790942 2550 scope.go:117] "RemoveContainer" containerID="4ad22071e4f91e951c0a43015825ecc2ff2dad2195fbe7b4c8825e077750c559" Jul 30 18:18:30.852902 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.852843 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:18:30.961345 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961315 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d571fd4-6304-48be-8076-427216ccf88d-istiod-ca-cert\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961360 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-istio-token\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961409 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw5d8\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-kube-api-access-gw5d8\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961428 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-certs\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961449 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-credential-socket\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961544 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961473 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d571fd4-6304-48be-8076-427216ccf88d-istio-podinfo\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961882 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961550 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-data\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961882 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961579 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-socket\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961882 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961608 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-envoy\") pod \"9d571fd4-6304-48be-8076-427216ccf88d\" (UID: \"9d571fd4-6304-48be-8076-427216ccf88d\") " Jul 30 18:18:30.961882 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961794 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:30.962102 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961880 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d571fd4-6304-48be-8076-427216ccf88d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:18:30.962102 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.961992 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d571fd4-6304-48be-8076-427216ccf88d-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:30.962102 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.962011 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:30.962261 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.962211 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:30.962261 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.962231 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:30.962629 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.962584 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:30.964195 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.964161 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:18:30.964339 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.964316 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d571fd4-6304-48be-8076-427216ccf88d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:18:30.964426 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.964316 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:18:30.964426 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:30.964348 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-kube-api-access-gw5d8" (OuterVolumeSpecName: "kube-api-access-gw5d8") pod "9d571fd4-6304-48be-8076-427216ccf88d" (UID: "9d571fd4-6304-48be-8076-427216ccf88d"). InnerVolumeSpecName "kube-api-access-gw5d8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:18:31.063436 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063409 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.063436 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063432 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.063436 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063442 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.063637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063449 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.063637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063457 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw5d8\" (UniqueName: \"kubernetes.io/projected/9d571fd4-6304-48be-8076-427216ccf88d-kube-api-access-gw5d8\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.063637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063464 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d571fd4-6304-48be-8076-427216ccf88d-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.063637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.063472 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d571fd4-6304-48be-8076-427216ccf88d-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:18:31.796698 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.796669 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" Jul 30 18:18:31.797189 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.796702 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6" event={"ID":"9d571fd4-6304-48be-8076-427216ccf88d","Type":"ContainerDied","Data":"58da4b77e639d07cb762b3268c52ebd1a33a9b14a69fda47c976e3d66735ed71"} Jul 30 18:18:31.797189 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.796754 2550 scope.go:117] "RemoveContainer" containerID="8ab6967a5207cdbfc94aabde5423f16c74581fbd9e8a3c717bfa698d7f53743b" Jul 30 18:18:31.820354 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.820327 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6"] Jul 30 18:18:31.825194 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:31.825172 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b51ede-openshift-default-74f86b97fd-hhmq6"] Jul 30 18:18:33.808738 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:33.808704 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d571fd4-6304-48be-8076-427216ccf88d" path="/var/lib/kubelet/pods/9d571fd4-6304-48be-8076-427216ccf88d/volumes" Jul 30 18:18:44.783555 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.783517 2550 patch_prober.go:28] interesting pod/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 18:18:44.783988 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.783570 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 18:18:44.877461 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.877423 2550 generic.go:356] "Generic (PLEG): container finished" podID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerID="5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9" exitCode=0 Jul 30 18:18:44.877628 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.877491 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" event={"ID":"f8555d7d-14f9-481d-8ee3-c09b188b0e34","Type":"ContainerDied","Data":"5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9"} Jul 30 18:18:44.877939 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.877919 2550 scope.go:117] "RemoveContainer" containerID="5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9" Jul 30 18:18:44.878913 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.878888 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:44.878971 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.878956 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:44.879014 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:44.878988 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:45.884940 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:45.884905 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" event={"ID":"f8555d7d-14f9-481d-8ee3-c09b188b0e34","Type":"ContainerStarted","Data":"613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e"} Jul 30 18:18:46.834779 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:46.834741 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:46.835234 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:46.835110 2550 patch_prober.go:28] interesting pod/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 18:18:46.835404 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:46.835379 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 18:18:47.834215 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:47.834174 2550 patch_prober.go:28] interesting pod/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 18:18:47.834731 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:47.834245 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 18:18:48.834921 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:48.834866 2550 patch_prober.go:28] interesting pod/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 18:18:48.835417 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:48.834942 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 18:18:49.838445 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:49.838415 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:49.838887 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:49.838665 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:49.839360 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:49.839342 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:18:55.399653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.399615 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb"] Jul 30 18:18:55.400198 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.400177 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" Jul 30 18:18:55.400255 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.400203 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" Jul 30 18:18:55.400255 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.400221 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" Jul 30 18:18:55.400255 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.400230 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1041b4ab-bf9f-45bb-82fa-616b32ba3003" containerName="istio-proxy" Jul 30 18:18:55.400409 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.400336 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" Jul 30 18:18:55.400409 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.400350 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" Jul 30 18:18:55.405320 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.405302 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.408196 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.408172 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fbbaa9-openshift-default-dockercfg-wwtgf\"" Jul 30 18:18:55.416002 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.415977 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb"] Jul 30 18:18:55.474221 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474188 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-socket\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474236 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-token\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474264 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-podinfo\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474342 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-data\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474372 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-credential-socket\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474396 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-envoy\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474428 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474426 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istiod-ca-cert\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474667 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474456 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-certs\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.474667 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.474499 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4npzw\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-kube-api-access-4npzw\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575061 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575027 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4npzw\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-kube-api-access-4npzw\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575090 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-socket\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575118 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-token\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575135 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-podinfo\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575166 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-data\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575186 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-credential-socket\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575206 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-envoy\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575236 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istiod-ca-cert\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575272 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575266 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-certs\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575447 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-socket\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575516 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-data\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575566 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-certs\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.575722 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575692 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-credential-socket\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.576008 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.575988 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istiod-ca-cert\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.577769 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.577743 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-envoy\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.577928 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.577908 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-podinfo\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.584270 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.584241 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-token\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.584358 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.584302 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4npzw\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-kube-api-access-4npzw\") pod \"e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.717157 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.717064 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:55.884531 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.884506 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb"] Jul 30 18:18:55.886389 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:18:55.886358 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9cc46fa_ff60_4813_9c9f_69a7ced78ff3.slice/crio-c59d7fb1cccd40cf82d1868d1258696c4eb5187a41243eca6526c3a487d78947 WatchSource:0}: Error finding container c59d7fb1cccd40cf82d1868d1258696c4eb5187a41243eca6526c3a487d78947: Status 404 returned error can't find the container with id c59d7fb1cccd40cf82d1868d1258696c4eb5187a41243eca6526c3a487d78947 Jul 30 18:18:55.888505 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.888472 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:55.888586 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.888531 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:55.888586 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.888558 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:18:55.940065 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:55.940039 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" event={"ID":"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3","Type":"ContainerStarted","Data":"c59d7fb1cccd40cf82d1868d1258696c4eb5187a41243eca6526c3a487d78947"} Jul 30 18:18:56.947338 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:56.947296 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" event={"ID":"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3","Type":"ContainerStarted","Data":"62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70"} Jul 30 18:18:56.967499 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:56.967456 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podStartSLOduration=1.9674414580000001 podStartE2EDuration="1.967441458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:18:56.967026562 +0000 UTC m=+3131.699541247" watchObservedRunningTime="2026-07-30 18:18:56.967441458 +0000 UTC m=+3131.699956140" Jul 30 18:18:57.717233 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:57.717199 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:57.722695 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:57.722667 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:57.952263 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:57.952229 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:18:57.953265 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:18:57.953247 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:19:00.616825 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.616791 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47"] Jul 30 18:19:00.617197 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.617177 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" Jul 30 18:19:00.617197 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.617190 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d571fd4-6304-48be-8076-427216ccf88d" containerName="istio-proxy" Jul 30 18:19:00.620271 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.620255 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.622910 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.622887 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-fbbaa9-dockercfg-vlp49\"" Jul 30 18:19:00.623026 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.622912 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-fbbaa9\"" Jul 30 18:19:00.630745 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.630725 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47"] Jul 30 18:19:00.727877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.727798 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9530180-cfb3-48dd-801b-a3a007b1ec7f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.728059 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.727950 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knrsz\" (UniqueName: \"kubernetes.io/projected/c9530180-cfb3-48dd-801b-a3a007b1ec7f-kube-api-access-knrsz\") pod \"payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.829404 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.829371 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knrsz\" (UniqueName: \"kubernetes.io/projected/c9530180-cfb3-48dd-801b-a3a007b1ec7f-kube-api-access-knrsz\") pod \"payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.829569 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.829431 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9530180-cfb3-48dd-801b-a3a007b1ec7f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.830102 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.830084 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9530180-cfb3-48dd-801b-a3a007b1ec7f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.838853 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.838796 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knrsz\" (UniqueName: \"kubernetes.io/projected/c9530180-cfb3-48dd-801b-a3a007b1ec7f-kube-api-access-knrsz\") pod \"payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:00.931262 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:00.931197 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:01.276336 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:01.276311 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47"] Jul 30 18:19:01.277492 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:19:01.277470 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9530180_cfb3_48dd_801b_a3a007b1ec7f.slice/crio-cdcf35a71448b41687b5adac11dafd2bb2b890bc56ca854ebdce7621a87fc03a WatchSource:0}: Error finding container cdcf35a71448b41687b5adac11dafd2bb2b890bc56ca854ebdce7621a87fc03a: Status 404 returned error can't find the container with id cdcf35a71448b41687b5adac11dafd2bb2b890bc56ca854ebdce7621a87fc03a Jul 30 18:19:01.975653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:01.975615 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" event={"ID":"c9530180-cfb3-48dd-801b-a3a007b1ec7f","Type":"ContainerStarted","Data":"f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed"} Jul 30 18:19:01.976071 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:01.975672 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" event={"ID":"c9530180-cfb3-48dd-801b-a3a007b1ec7f","Type":"ContainerStarted","Data":"cdcf35a71448b41687b5adac11dafd2bb2b890bc56ca854ebdce7621a87fc03a"} Jul 30 18:19:01.976071 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:01.975768 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:01.993401 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:01.993357 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" podStartSLOduration=1.993341888 podStartE2EDuration="1.993341888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:19:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:19:01.992235321 +0000 UTC m=+3136.724750005" watchObservedRunningTime="2026-07-30 18:19:01.993341888 +0000 UTC m=+3136.725856572" Jul 30 18:19:12.983680 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:12.983645 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:19:13.994946 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:13.994914 2550 patch_prober.go:28] interesting pod/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 18:19:13.995318 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:13.994969 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 18:19:15.068055 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:15.067964 2550 generic.go:356] "Generic (PLEG): container finished" podID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerID="62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70" exitCode=0 Jul 30 18:19:15.068055 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:15.068035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" event={"ID":"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3","Type":"ContainerDied","Data":"62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70"} Jul 30 18:19:15.068489 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:15.068418 2550 scope.go:117] "RemoveContainer" containerID="62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70" Jul 30 18:19:15.069495 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:15.069467 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:19:15.069608 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:15.069536 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:19:15.069608 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:15.069566 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:19:16.075045 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:16.075012 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" event={"ID":"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3","Type":"ContainerStarted","Data":"cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786"} Jul 30 18:19:16.717873 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:16.717840 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:19:16.718067 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:16.718045 2550 patch_prober.go:28] interesting pod/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 18:19:16.718113 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:16.718099 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 18:19:17.718384 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:17.718351 2550 patch_prober.go:28] interesting pod/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 18:19:17.718779 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:17.718400 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 18:19:18.718440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:18.718404 2550 patch_prober.go:28] interesting pod/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 18:19:18.718862 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:18.718461 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 18:19:19.717618 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:19.717584 2550 patch_prober.go:28] interesting pod/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 18:19:19.717793 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:19.717649 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 18:19:20.721451 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:20.721423 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:19:20.721866 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:20.721685 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:19:20.722385 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:19:20.722369 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:22:46.083163 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:22:46.083129 2550 scope.go:117] "RemoveContainer" containerID="7f825188ea3ee9e1da8c8d12d7b7ac74beea494ff0e8cdae59826f46dfaaa97f" Jul 30 18:29:50.252634 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.252597 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55"] Jul 30 18:29:50.256447 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.256423 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.259979 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.259961 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-fadb12-openshift-default-dockercfg-mkqqj\"" Jul 30 18:29:50.269959 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.269936 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55"] Jul 30 18:29:50.378775 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.378741 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-data\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.378785 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-credential-socket\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.378835 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2257821-4879-4e10-9779-b33e21d82b99-istio-podinfo\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.378907 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-socket\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379009 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.378993 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-certs\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379240 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.379055 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-envoy\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379240 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.379088 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-istio-token\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379240 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.379115 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np7dd\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-kube-api-access-np7dd\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.379240 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.379152 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2257821-4879-4e10-9779-b33e21d82b99-istiod-ca-cert\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480238 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2257821-4879-4e10-9779-b33e21d82b99-istio-podinfo\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480277 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-socket\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480296 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-certs\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480331 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-envoy\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480349 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-istio-token\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480366 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np7dd\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-kube-api-access-np7dd\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480387 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2257821-4879-4e10-9779-b33e21d82b99-istiod-ca-cert\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480437 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-data\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480534 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480488 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-credential-socket\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480912 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480684 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-socket\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.480912 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480800 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-data\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.481017 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.480947 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-certs\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.481069 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.481028 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-credential-socket\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.481287 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.481268 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2257821-4879-4e10-9779-b33e21d82b99-istiod-ca-cert\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.482652 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.482633 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-envoy\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.483133 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.483111 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2257821-4879-4e10-9779-b33e21d82b99-istio-podinfo\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.489162 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.489135 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-istio-token\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.489440 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.489421 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np7dd\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-kube-api-access-np7dd\") pod \"e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.569435 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.569351 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:50.718210 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.718186 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55"] Jul 30 18:29:50.719726 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:29:50.719703 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2257821_4879_4e10_9779_b33e21d82b99.slice/crio-188bccacd2731948c453a7cdc62812ccf5e28ae7e7d720a4621227a512cef324 WatchSource:0}: Error finding container 188bccacd2731948c453a7cdc62812ccf5e28ae7e7d720a4621227a512cef324: Status 404 returned error can't find the container with id 188bccacd2731948c453a7cdc62812ccf5e28ae7e7d720a4621227a512cef324 Jul 30 18:29:50.722111 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.722080 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:29:50.722201 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.722140 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:29:50.722201 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:50.722167 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:29:51.410156 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:51.410126 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" event={"ID":"a2257821-4879-4e10-9779-b33e21d82b99","Type":"ContainerStarted","Data":"ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96"} Jul 30 18:29:51.410565 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:51.410174 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" event={"ID":"a2257821-4879-4e10-9779-b33e21d82b99","Type":"ContainerStarted","Data":"188bccacd2731948c453a7cdc62812ccf5e28ae7e7d720a4621227a512cef324"} Jul 30 18:29:51.435748 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:51.435701 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" podStartSLOduration=1.435685667 podStartE2EDuration="1.435685667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:29:51.433356903 +0000 UTC m=+3786.165871586" watchObservedRunningTime="2026-07-30 18:29:51.435685667 +0000 UTC m=+3786.168200352" Jul 30 18:29:51.570513 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:51.570475 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:51.575446 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:51.575423 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:52.415835 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:52.415775 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:29:52.416731 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:29:52.416714 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:30:10.510438 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:10.510405 2550 generic.go:356] "Generic (PLEG): container finished" podID="a2257821-4879-4e10-9779-b33e21d82b99" containerID="ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96" exitCode=0 Jul 30 18:30:10.510831 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:10.510469 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" event={"ID":"a2257821-4879-4e10-9779-b33e21d82b99","Type":"ContainerDied","Data":"ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96"} Jul 30 18:30:10.510993 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:10.510979 2550 scope.go:117] "RemoveContainer" containerID="ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96" Jul 30 18:30:10.512096 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:10.512071 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:30:10.512177 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:10.512138 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:30:10.512177 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:10.512169 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 30 18:30:11.517504 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:11.517464 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" event={"ID":"a2257821-4879-4e10-9779-b33e21d82b99","Type":"ContainerStarted","Data":"f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903"} Jul 30 18:30:11.569637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:11.569606 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:30:11.569954 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:11.569929 2550 patch_prober.go:28] interesting pod/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 18:30:11.570068 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:11.569983 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 18:30:12.570637 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:12.570606 2550 patch_prober.go:28] interesting pod/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 18:30:12.571107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:12.570659 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 18:30:13.569881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:13.569843 2550 patch_prober.go:28] interesting pod/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 18:30:13.570065 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:13.569897 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 18:30:14.569756 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:14.569718 2550 patch_prober.go:28] interesting pod/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 18:30:14.570194 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:14.569778 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 18:30:15.573251 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:15.573221 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:30:15.573635 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:15.573551 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:30:15.574265 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:15.574243 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:30:26.962405 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:26.962332 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l"] Jul 30 18:30:26.964967 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:26.964951 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:26.967976 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:26.967954 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c8d7de\"" Jul 30 18:30:26.978931 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:26.978912 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l"] Jul 30 18:30:27.026777 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.026745 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af1ce589-bc7d-4d7b-81fc-e1bed390891b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.026777 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.026779 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mb9s\" (UniqueName: \"kubernetes.io/projected/af1ce589-bc7d-4d7b-81fc-e1bed390891b-kube-api-access-8mb9s\") pod \"payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.128151 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.128114 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af1ce589-bc7d-4d7b-81fc-e1bed390891b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.128151 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.128154 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mb9s\" (UniqueName: \"kubernetes.io/projected/af1ce589-bc7d-4d7b-81fc-e1bed390891b-kube-api-access-8mb9s\") pod \"payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.128842 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.128806 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af1ce589-bc7d-4d7b-81fc-e1bed390891b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.137252 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.137231 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mb9s\" (UniqueName: \"kubernetes.io/projected/af1ce589-bc7d-4d7b-81fc-e1bed390891b-kube-api-access-8mb9s\") pod \"payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.276207 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.276117 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.414454 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.414427 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l"] Jul 30 18:30:27.416453 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:30:27.416423 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1ce589_bc7d_4d7b_81fc_e1bed390891b.slice/crio-fd4d4fe9854ed994c36c7ec00edd7df876d5412c7180b58d8e1b438d4f1055f8 WatchSource:0}: Error finding container fd4d4fe9854ed994c36c7ec00edd7df876d5412c7180b58d8e1b438d4f1055f8: Status 404 returned error can't find the container with id fd4d4fe9854ed994c36c7ec00edd7df876d5412c7180b58d8e1b438d4f1055f8 Jul 30 18:30:27.616042 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.616006 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" event={"ID":"af1ce589-bc7d-4d7b-81fc-e1bed390891b","Type":"ContainerStarted","Data":"06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639"} Jul 30 18:30:27.616239 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.616058 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" event={"ID":"af1ce589-bc7d-4d7b-81fc-e1bed390891b","Type":"ContainerStarted","Data":"fd4d4fe9854ed994c36c7ec00edd7df876d5412c7180b58d8e1b438d4f1055f8"} Jul 30 18:30:27.616239 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.616131 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:30:27.634360 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:27.634310 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" podStartSLOduration=1.63429417 podStartE2EDuration="1.63429417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:30:27.632334424 +0000 UTC m=+3822.364849107" watchObservedRunningTime="2026-07-30 18:30:27.63429417 +0000 UTC m=+3822.366808892" Jul 30 18:30:38.623583 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:38.623539 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" podUID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.59:9004: connect: connection refused" Jul 30 18:30:48.623442 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:30:48.623411 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:31:53.225772 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.225729 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg"] Jul 30 18:31:53.229318 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.229297 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.232138 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.232111 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-46b4e2-dockercfg-6rw4z\"" Jul 30 18:31:53.232649 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.232627 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-46b4e2\"" Jul 30 18:31:53.239266 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.238987 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg"] Jul 30 18:31:53.251182 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.251157 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd"] Jul 30 18:31:53.254509 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.254490 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.265563 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.265534 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd"] Jul 30 18:31:53.342744 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.342699 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngwsb\" (UniqueName: \"kubernetes.io/projected/e868d194-5090-45a7-9ae9-4ddc9e99bde0-kube-api-access-ngwsb\") pod \"payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.343118 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.343097 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e868d194-5090-45a7-9ae9-4ddc9e99bde0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.343261 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.343245 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfk2t\" (UniqueName: \"kubernetes.io/projected/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-kube-api-access-tfk2t\") pod \"payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.343402 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.343389 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.444339 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.444300 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.444519 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.444401 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngwsb\" (UniqueName: \"kubernetes.io/projected/e868d194-5090-45a7-9ae9-4ddc9e99bde0-kube-api-access-ngwsb\") pod \"payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.444519 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.444474 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e868d194-5090-45a7-9ae9-4ddc9e99bde0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.444519 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.444502 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfk2t\" (UniqueName: \"kubernetes.io/projected/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-kube-api-access-tfk2t\") pod \"payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.445204 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.445186 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e868d194-5090-45a7-9ae9-4ddc9e99bde0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.445252 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.445187 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.453497 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.453470 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfk2t\" (UniqueName: \"kubernetes.io/projected/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-kube-api-access-tfk2t\") pod \"payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.453699 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.453675 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngwsb\" (UniqueName: \"kubernetes.io/projected/e868d194-5090-45a7-9ae9-4ddc9e99bde0-kube-api-access-ngwsb\") pod \"payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.542039 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.541946 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:53.566433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.566407 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:53.699570 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.699538 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg"] Jul 30 18:31:53.699853 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:31:53.699804 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode868d194_5090_45a7_9ae9_4ddc9e99bde0.slice/crio-e7629ef6215ac371dd26c44faa65f1d116f5e052ff73bfaf60f7980fa190f748 WatchSource:0}: Error finding container e7629ef6215ac371dd26c44faa65f1d116f5e052ff73bfaf60f7980fa190f748: Status 404 returned error can't find the container with id e7629ef6215ac371dd26c44faa65f1d116f5e052ff73bfaf60f7980fa190f748 Jul 30 18:31:53.721122 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:53.721097 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd"] Jul 30 18:31:53.723319 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:31:53.723296 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc963d76_51e3_4abd_b5cd_4b40a9df0c65.slice/crio-cefb96135b2523381213c3cc2f323bf1d2a35204c12b8abbd0435975da4b7095 WatchSource:0}: Error finding container cefb96135b2523381213c3cc2f323bf1d2a35204c12b8abbd0435975da4b7095: Status 404 returned error can't find the container with id cefb96135b2523381213c3cc2f323bf1d2a35204c12b8abbd0435975da4b7095 Jul 30 18:31:54.098485 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.098452 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" event={"ID":"e868d194-5090-45a7-9ae9-4ddc9e99bde0","Type":"ContainerStarted","Data":"0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212"} Jul 30 18:31:54.098699 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.098521 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" event={"ID":"e868d194-5090-45a7-9ae9-4ddc9e99bde0","Type":"ContainerStarted","Data":"e7629ef6215ac371dd26c44faa65f1d116f5e052ff73bfaf60f7980fa190f748"} Jul 30 18:31:54.098699 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.098647 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:31:54.100040 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.100007 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" event={"ID":"bc963d76-51e3-4abd-b5cd-4b40a9df0c65","Type":"ContainerStarted","Data":"82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60"} Jul 30 18:31:54.100040 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.100044 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" event={"ID":"bc963d76-51e3-4abd-b5cd-4b40a9df0c65","Type":"ContainerStarted","Data":"cefb96135b2523381213c3cc2f323bf1d2a35204c12b8abbd0435975da4b7095"} Jul 30 18:31:54.100204 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.100070 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:31:54.116877 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.116829 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" podStartSLOduration=1.11679762 podStartE2EDuration="1.11679762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:31:54.114773348 +0000 UTC m=+3908.847288039" watchObservedRunningTime="2026-07-30 18:31:54.11679762 +0000 UTC m=+3908.849312302" Jul 30 18:31:54.134242 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:31:54.134186 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" podStartSLOduration=1.134166185 podStartE2EDuration="1.134166185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:31:54.130983657 +0000 UTC m=+3908.863498362" watchObservedRunningTime="2026-07-30 18:31:54.134166185 +0000 UTC m=+3908.866680872" Jul 30 18:32:05.108936 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:05.108900 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:32:05.109393 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:05.108979 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:32:23.361652 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.361611 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd"] Jul 30 18:32:23.362649 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.362596 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" podUID="bc963d76-51e3-4abd-b5cd-4b40a9df0c65" containerName="payload-processing" containerID="cri-o://82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60" gracePeriod=30 Jul 30 18:32:23.366372 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.366345 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg"] Jul 30 18:32:23.366624 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.366581 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" podUID="e868d194-5090-45a7-9ae9-4ddc9e99bde0" containerName="payload-pre-processing" containerID="cri-o://0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212" gracePeriod=30 Jul 30 18:32:23.644709 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.644678 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:32:23.682095 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.682069 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:32:23.730590 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.730554 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngwsb\" (UniqueName: \"kubernetes.io/projected/e868d194-5090-45a7-9ae9-4ddc9e99bde0-kube-api-access-ngwsb\") pod \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " Jul 30 18:32:23.730785 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.730639 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-plugins-config-volume\") pod \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " Jul 30 18:32:23.730785 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.730724 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e868d194-5090-45a7-9ae9-4ddc9e99bde0-plugins-config-volume\") pod \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\" (UID: \"e868d194-5090-45a7-9ae9-4ddc9e99bde0\") " Jul 30 18:32:23.730785 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.730768 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfk2t\" (UniqueName: \"kubernetes.io/projected/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-kube-api-access-tfk2t\") pod \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\" (UID: \"bc963d76-51e3-4abd-b5cd-4b40a9df0c65\") " Jul 30 18:32:23.731047 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.731027 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc963d76-51e3-4abd-b5cd-4b40a9df0c65" (UID: "bc963d76-51e3-4abd-b5cd-4b40a9df0c65"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:32:23.731114 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.731060 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e868d194-5090-45a7-9ae9-4ddc9e99bde0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e868d194-5090-45a7-9ae9-4ddc9e99bde0" (UID: "e868d194-5090-45a7-9ae9-4ddc9e99bde0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:32:23.731180 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.731141 2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-plugins-config-volume\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:32:23.731180 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.731152 2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e868d194-5090-45a7-9ae9-4ddc9e99bde0-plugins-config-volume\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:32:23.733015 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.732988 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e868d194-5090-45a7-9ae9-4ddc9e99bde0-kube-api-access-ngwsb" (OuterVolumeSpecName: "kube-api-access-ngwsb") pod "e868d194-5090-45a7-9ae9-4ddc9e99bde0" (UID: "e868d194-5090-45a7-9ae9-4ddc9e99bde0"). InnerVolumeSpecName "kube-api-access-ngwsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:32:23.733151 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.733025 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-kube-api-access-tfk2t" (OuterVolumeSpecName: "kube-api-access-tfk2t") pod "bc963d76-51e3-4abd-b5cd-4b40a9df0c65" (UID: "bc963d76-51e3-4abd-b5cd-4b40a9df0c65"). InnerVolumeSpecName "kube-api-access-tfk2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:32:23.832080 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.832049 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfk2t\" (UniqueName: \"kubernetes.io/projected/bc963d76-51e3-4abd-b5cd-4b40a9df0c65-kube-api-access-tfk2t\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:32:23.832080 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:23.832081 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngwsb\" (UniqueName: \"kubernetes.io/projected/e868d194-5090-45a7-9ae9-4ddc9e99bde0-kube-api-access-ngwsb\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:32:24.268495 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.268461 2550 generic.go:356] "Generic (PLEG): container finished" podID="e868d194-5090-45a7-9ae9-4ddc9e99bde0" containerID="0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212" exitCode=0 Jul 30 18:32:24.268700 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.268511 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" Jul 30 18:32:24.268700 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.268555 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" event={"ID":"e868d194-5090-45a7-9ae9-4ddc9e99bde0","Type":"ContainerDied","Data":"0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212"} Jul 30 18:32:24.268700 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.268616 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg" event={"ID":"e868d194-5090-45a7-9ae9-4ddc9e99bde0","Type":"ContainerDied","Data":"e7629ef6215ac371dd26c44faa65f1d116f5e052ff73bfaf60f7980fa190f748"} Jul 30 18:32:24.268700 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.268645 2550 scope.go:117] "RemoveContainer" containerID="0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212" Jul 30 18:32:24.270252 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.270229 2550 generic.go:356] "Generic (PLEG): container finished" podID="bc963d76-51e3-4abd-b5cd-4b40a9df0c65" containerID="82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60" exitCode=0 Jul 30 18:32:24.270359 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.270277 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" Jul 30 18:32:24.270359 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.270286 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" event={"ID":"bc963d76-51e3-4abd-b5cd-4b40a9df0c65","Type":"ContainerDied","Data":"82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60"} Jul 30 18:32:24.270359 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.270327 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd" event={"ID":"bc963d76-51e3-4abd-b5cd-4b40a9df0c65","Type":"ContainerDied","Data":"cefb96135b2523381213c3cc2f323bf1d2a35204c12b8abbd0435975da4b7095"} Jul 30 18:32:24.280356 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.280308 2550 scope.go:117] "RemoveContainer" containerID="0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212" Jul 30 18:32:24.280753 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:32:24.280728 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212\": container with ID starting with 0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212 not found: ID does not exist" containerID="0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212" Jul 30 18:32:24.280887 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.280765 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212"} err="failed to get container status \"0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212\": rpc error: code = NotFound desc = could not find container \"0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212\": container with ID starting with 0987b2decd678bd55a855b53ffd167e6d86a6a847995c28f363ca61985277212 not found: ID does not exist" Jul 30 18:32:24.280887 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.280785 2550 scope.go:117] "RemoveContainer" containerID="82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60" Jul 30 18:32:24.289584 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.289561 2550 scope.go:117] "RemoveContainer" containerID="82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60" Jul 30 18:32:24.289917 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:32:24.289876 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60\": container with ID starting with 82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60 not found: ID does not exist" containerID="82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60" Jul 30 18:32:24.290122 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.289921 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60"} err="failed to get container status \"82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60\": rpc error: code = NotFound desc = could not find container \"82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60\": container with ID starting with 82f6d88691148521e40cfe67f15de59ebaec8f367b716097adedf522e25d5f60 not found: ID does not exist" Jul 30 18:32:24.295778 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.295751 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg"] Jul 30 18:32:24.300084 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.300057 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-46b4e2-6f8dd7bdd4-nxvkg"] Jul 30 18:32:24.310189 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.310160 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd"] Jul 30 18:32:24.314897 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:24.314868 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-46b4e2-859f4cdfd6-lf4bd"] Jul 30 18:32:25.815187 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:25.815151 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc963d76-51e3-4abd-b5cd-4b40a9df0c65" path="/var/lib/kubelet/pods/bc963d76-51e3-4abd-b5cd-4b40a9df0c65/volumes" Jul 30 18:32:25.815634 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:32:25.815505 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e868d194-5090-45a7-9ae9-4ddc9e99bde0" path="/var/lib/kubelet/pods/e868d194-5090-45a7-9ae9-4ddc9e99bde0/volumes" Jul 30 18:33:38.820397 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:38.820359 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55"] Jul 30 18:33:38.820977 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:38.820619 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" containerID="cri-o://f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903" gracePeriod=30 Jul 30 18:33:44.072657 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.072627 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:33:44.106139 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106115 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-envoy\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106321 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106202 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2257821-4879-4e10-9779-b33e21d82b99-istio-podinfo\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106321 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106224 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-istio-token\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106321 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106245 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-credential-socket\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106321 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106261 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-certs\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106321 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106304 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-socket\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106579 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106357 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np7dd\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-kube-api-access-np7dd\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106579 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106387 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2257821-4879-4e10-9779-b33e21d82b99-istiod-ca-cert\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106579 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106421 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-data\") pod \"a2257821-4879-4e10-9779-b33e21d82b99\" (UID: \"a2257821-4879-4e10-9779-b33e21d82b99\") " Jul 30 18:33:44.106579 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106520 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:33:44.106789 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106745 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.106881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.106842 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:33:44.107087 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.107028 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:33:44.107229 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.107181 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2257821-4879-4e10-9779-b33e21d82b99-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:33:44.107641 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.107617 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:33:44.108543 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.108512 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:33:44.108656 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.108614 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:33:44.108857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.108838 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2257821-4879-4e10-9779-b33e21d82b99-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:33:44.108927 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.108899 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-kube-api-access-np7dd" (OuterVolumeSpecName: "kube-api-access-np7dd") pod "a2257821-4879-4e10-9779-b33e21d82b99" (UID: "a2257821-4879-4e10-9779-b33e21d82b99"). InnerVolumeSpecName "kube-api-access-np7dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:33:44.207829 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207777 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2257821-4879-4e10-9779-b33e21d82b99-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.207829 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207800 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.207829 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207829 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.207829 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207838 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.208096 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207847 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np7dd\" (UniqueName: \"kubernetes.io/projected/a2257821-4879-4e10-9779-b33e21d82b99-kube-api-access-np7dd\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.208096 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207855 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2257821-4879-4e10-9779-b33e21d82b99-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.208096 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207864 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.208096 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.207870 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2257821-4879-4e10-9779-b33e21d82b99-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:44.713005 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.712966 2550 generic.go:356] "Generic (PLEG): container finished" podID="a2257821-4879-4e10-9779-b33e21d82b99" containerID="f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903" exitCode=0 Jul 30 18:33:44.713206 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.713018 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" Jul 30 18:33:44.713206 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.713049 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" event={"ID":"a2257821-4879-4e10-9779-b33e21d82b99","Type":"ContainerDied","Data":"f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903"} Jul 30 18:33:44.713206 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.713108 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55" event={"ID":"a2257821-4879-4e10-9779-b33e21d82b99","Type":"ContainerDied","Data":"188bccacd2731948c453a7cdc62812ccf5e28ae7e7d720a4621227a512cef324"} Jul 30 18:33:44.713206 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.713135 2550 scope.go:117] "RemoveContainer" containerID="f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903" Jul 30 18:33:44.722601 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.722585 2550 scope.go:117] "RemoveContainer" containerID="ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96" Jul 30 18:33:44.730572 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.730555 2550 scope.go:117] "RemoveContainer" containerID="f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903" Jul 30 18:33:44.730876 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:33:44.730855 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903\": container with ID starting with f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903 not found: ID does not exist" containerID="f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903" Jul 30 18:33:44.730959 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.730884 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903"} err="failed to get container status \"f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903\": rpc error: code = NotFound desc = could not find container \"f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903\": container with ID starting with f84338559eb26e9b48cfa5935930b93bff823c835a5fae7300aecf1dbe3fe903 not found: ID does not exist" Jul 30 18:33:44.730959 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.730901 2550 scope.go:117] "RemoveContainer" containerID="ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96" Jul 30 18:33:44.731339 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:33:44.731308 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96\": container with ID starting with ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96 not found: ID does not exist" containerID="ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96" Jul 30 18:33:44.731446 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.731346 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96"} err="failed to get container status \"ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96\": rpc error: code = NotFound desc = could not find container \"ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96\": container with ID starting with ce6b8bc72a7bd1a1ff7c7fc9d5ac047193890148a7777dbaac2e979fdfac5b96 not found: ID does not exist" Jul 30 18:33:44.734369 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.734347 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55"] Jul 30 18:33:44.738073 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:44.738053 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-fadb12-openshift-default-c9464c7d-7wp55"] Jul 30 18:33:45.809556 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:45.809525 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2257821-4879-4e10-9779-b33e21d82b99" path="/var/lib/kubelet/pods/a2257821-4879-4e10-9779-b33e21d82b99/volumes" Jul 30 18:33:49.689371 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:49.689338 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l"] Jul 30 18:33:49.689884 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:49.689585 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" podUID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" containerName="payload-pre-processing" containerID="cri-o://06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639" gracePeriod=30 Jul 30 18:33:49.951833 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:49.951754 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:33:50.057924 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.057892 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mb9s\" (UniqueName: \"kubernetes.io/projected/af1ce589-bc7d-4d7b-81fc-e1bed390891b-kube-api-access-8mb9s\") pod \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " Jul 30 18:33:50.058112 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.057999 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af1ce589-bc7d-4d7b-81fc-e1bed390891b-plugins-config-volume\") pod \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\" (UID: \"af1ce589-bc7d-4d7b-81fc-e1bed390891b\") " Jul 30 18:33:50.058365 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.058342 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af1ce589-bc7d-4d7b-81fc-e1bed390891b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "af1ce589-bc7d-4d7b-81fc-e1bed390891b" (UID: "af1ce589-bc7d-4d7b-81fc-e1bed390891b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:33:50.060236 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.060215 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af1ce589-bc7d-4d7b-81fc-e1bed390891b-kube-api-access-8mb9s" (OuterVolumeSpecName: "kube-api-access-8mb9s") pod "af1ce589-bc7d-4d7b-81fc-e1bed390891b" (UID: "af1ce589-bc7d-4d7b-81fc-e1bed390891b"). InnerVolumeSpecName "kube-api-access-8mb9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:33:50.158910 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.158877 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mb9s\" (UniqueName: \"kubernetes.io/projected/af1ce589-bc7d-4d7b-81fc-e1bed390891b-kube-api-access-8mb9s\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:50.158910 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.158903 2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af1ce589-bc7d-4d7b-81fc-e1bed390891b-plugins-config-volume\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:33:50.749352 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.749314 2550 generic.go:356] "Generic (PLEG): container finished" podID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" containerID="06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639" exitCode=0 Jul 30 18:33:50.749710 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.749375 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" Jul 30 18:33:50.749710 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.749399 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" event={"ID":"af1ce589-bc7d-4d7b-81fc-e1bed390891b","Type":"ContainerDied","Data":"06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639"} Jul 30 18:33:50.749710 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.749477 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l" event={"ID":"af1ce589-bc7d-4d7b-81fc-e1bed390891b","Type":"ContainerDied","Data":"fd4d4fe9854ed994c36c7ec00edd7df876d5412c7180b58d8e1b438d4f1055f8"} Jul 30 18:33:50.749710 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.749506 2550 scope.go:117] "RemoveContainer" containerID="06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639" Jul 30 18:33:50.762277 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.762256 2550 scope.go:117] "RemoveContainer" containerID="06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639" Jul 30 18:33:50.762507 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:33:50.762488 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639\": container with ID starting with 06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639 not found: ID does not exist" containerID="06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639" Jul 30 18:33:50.762574 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.762523 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639"} err="failed to get container status \"06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639\": rpc error: code = NotFound desc = could not find container \"06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639\": container with ID starting with 06a17a0cda38d726888f4ad623d2e380b8592d467abd6c47c320e1ec218e4639 not found: ID does not exist" Jul 30 18:33:50.772215 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.772194 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l"] Jul 30 18:33:50.775895 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:50.775875 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c8d7de-6d487f7644-xc88l"] Jul 30 18:33:51.808607 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:33:51.808562 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" path="/var/lib/kubelet/pods/af1ce589-bc7d-4d7b-81fc-e1bed390891b/volumes" Jul 30 18:34:14.124555 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:14.124520 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l"] Jul 30 18:34:14.125183 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:14.124796 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" containerID="cri-o://613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e" gracePeriod=30 Jul 30 18:34:19.378036 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.378007 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:34:19.538549 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538455 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-credential-socket\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538549 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538505 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-data\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538549 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538541 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-socket\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538587 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-podinfo\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538607 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tks4s\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-kube-api-access-tks4s\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538646 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istiod-ca-cert\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538676 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-token\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538726 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538752 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-certs\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.538881 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538863 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-envoy\") pod \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\" (UID: \"f8555d7d-14f9-481d-8ee3-c09b188b0e34\") " Jul 30 18:34:19.539244 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.538920 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:19.539244 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.539028 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:19.539341 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.539246 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.539341 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.539265 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.539341 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.539279 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.539479 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.539333 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:34:19.539621 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.539595 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-data" (OuterVolumeSpecName: "istio-data") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:19.542152 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.541688 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:34:19.542152 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.541695 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-token" (OuterVolumeSpecName: "istio-token") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:34:19.542152 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.542060 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:19.542526 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.542166 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-kube-api-access-tks4s" (OuterVolumeSpecName: "kube-api-access-tks4s") pod "f8555d7d-14f9-481d-8ee3-c09b188b0e34" (UID: "f8555d7d-14f9-481d-8ee3-c09b188b0e34"). InnerVolumeSpecName "kube-api-access-tks4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:34:19.620347 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.620033 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47"] Jul 30 18:34:19.620347 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.620282 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" podUID="c9530180-cfb3-48dd-801b-a3a007b1ec7f" containerName="payload-processing" containerID="cri-o://f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed" gracePeriod=30 Jul 30 18:34:19.640317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.640287 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.640317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.640312 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.640317 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.640320 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.640605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.640328 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tks4s\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-kube-api-access-tks4s\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.640605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.640339 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.640605 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.640346 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8555d7d-14f9-481d-8ee3-c09b188b0e34-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:19.876753 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.876728 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:34:19.906311 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.906275 2550 generic.go:356] "Generic (PLEG): container finished" podID="c9530180-cfb3-48dd-801b-a3a007b1ec7f" containerID="f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed" exitCode=0 Jul 30 18:34:19.906491 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.906315 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" Jul 30 18:34:19.906491 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.906353 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" event={"ID":"c9530180-cfb3-48dd-801b-a3a007b1ec7f","Type":"ContainerDied","Data":"f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed"} Jul 30 18:34:19.906491 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.906404 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47" event={"ID":"c9530180-cfb3-48dd-801b-a3a007b1ec7f","Type":"ContainerDied","Data":"cdcf35a71448b41687b5adac11dafd2bb2b890bc56ca854ebdce7621a87fc03a"} Jul 30 18:34:19.906491 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.906432 2550 scope.go:117] "RemoveContainer" containerID="f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed" Jul 30 18:34:19.908255 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.908229 2550 generic.go:356] "Generic (PLEG): container finished" podID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerID="613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e" exitCode=0 Jul 30 18:34:19.908376 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.908326 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" Jul 30 18:34:19.908376 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.908356 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" event={"ID":"f8555d7d-14f9-481d-8ee3-c09b188b0e34","Type":"ContainerDied","Data":"613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e"} Jul 30 18:34:19.908458 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.908403 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l" event={"ID":"f8555d7d-14f9-481d-8ee3-c09b188b0e34","Type":"ContainerDied","Data":"cd311c6939c68db5425316dec49708fd5528cad0f45db192190e42aeda1f3f32"} Jul 30 18:34:19.915761 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.915742 2550 scope.go:117] "RemoveContainer" containerID="f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed" Jul 30 18:34:19.916022 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:34:19.916005 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed\": container with ID starting with f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed not found: ID does not exist" containerID="f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed" Jul 30 18:34:19.916099 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.916032 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed"} err="failed to get container status \"f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed\": rpc error: code = NotFound desc = could not find container \"f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed\": container with ID starting with f24bc7f92d2adbf6d054a0c863308edc0312c1ad1b283044f5b2c863f38fffed not found: ID does not exist" Jul 30 18:34:19.916099 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.916048 2550 scope.go:117] "RemoveContainer" containerID="613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e" Jul 30 18:34:19.925664 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.925642 2550 scope.go:117] "RemoveContainer" containerID="5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9" Jul 30 18:34:19.926690 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.926662 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l"] Jul 30 18:34:19.931101 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.931077 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8f586d-openshift-default-675868c86d-x9n9l"] Jul 30 18:34:19.935164 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.935146 2550 scope.go:117] "RemoveContainer" containerID="613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e" Jul 30 18:34:19.935503 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:34:19.935485 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e\": container with ID starting with 613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e not found: ID does not exist" containerID="613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e" Jul 30 18:34:19.935560 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.935510 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e"} err="failed to get container status \"613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e\": rpc error: code = NotFound desc = could not find container \"613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e\": container with ID starting with 613da7fa5b3b52a387fa9081194fef6275d8084f0f32ffe50d971c696995bf4e not found: ID does not exist" Jul 30 18:34:19.935560 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.935528 2550 scope.go:117] "RemoveContainer" containerID="5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9" Jul 30 18:34:19.935783 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:34:19.935768 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9\": container with ID starting with 5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9 not found: ID does not exist" containerID="5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9" Jul 30 18:34:19.935887 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:19.935787 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9"} err="failed to get container status \"5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9\": rpc error: code = NotFound desc = could not find container \"5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9\": container with ID starting with 5ce8561af5a4c22bae9149efcce51b11a718a023ba2c08f63f1fcbd99cafa4d9 not found: ID does not exist" Jul 30 18:34:20.045244 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.045198 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9530180-cfb3-48dd-801b-a3a007b1ec7f-plugins-config-volume\") pod \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " Jul 30 18:34:20.045446 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.045292 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knrsz\" (UniqueName: \"kubernetes.io/projected/c9530180-cfb3-48dd-801b-a3a007b1ec7f-kube-api-access-knrsz\") pod \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\" (UID: \"c9530180-cfb3-48dd-801b-a3a007b1ec7f\") " Jul 30 18:34:20.045572 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.045548 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9530180-cfb3-48dd-801b-a3a007b1ec7f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9530180-cfb3-48dd-801b-a3a007b1ec7f" (UID: "c9530180-cfb3-48dd-801b-a3a007b1ec7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:34:20.045713 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.045688 2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9530180-cfb3-48dd-801b-a3a007b1ec7f-plugins-config-volume\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:20.047445 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.047423 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9530180-cfb3-48dd-801b-a3a007b1ec7f-kube-api-access-knrsz" (OuterVolumeSpecName: "kube-api-access-knrsz") pod "c9530180-cfb3-48dd-801b-a3a007b1ec7f" (UID: "c9530180-cfb3-48dd-801b-a3a007b1ec7f"). InnerVolumeSpecName "kube-api-access-knrsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:34:20.146641 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.146613 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knrsz\" (UniqueName: \"kubernetes.io/projected/c9530180-cfb3-48dd-801b-a3a007b1ec7f-kube-api-access-knrsz\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:20.228332 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.228299 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47"] Jul 30 18:34:20.234226 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:20.234202 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fbbaa9-79c45c8db-sgl47"] Jul 30 18:34:21.808355 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:21.808311 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9530180-cfb3-48dd-801b-a3a007b1ec7f" path="/var/lib/kubelet/pods/c9530180-cfb3-48dd-801b-a3a007b1ec7f/volumes" Jul 30 18:34:21.808944 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:21.808924 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" path="/var/lib/kubelet/pods/f8555d7d-14f9-481d-8ee3-c09b188b0e34/volumes" Jul 30 18:34:26.469331 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:26.469296 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb"] Jul 30 18:34:26.469785 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:26.469502 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" containerID="cri-o://cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786" gracePeriod=30 Jul 30 18:34:31.727942 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.727916 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:34:31.748137 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748080 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-socket\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748137 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748115 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-certs\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748138 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-data\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748158 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-podinfo\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748175 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istiod-ca-cert\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748205 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-credential-socket\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748330 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748322 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-envoy\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748587 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748372 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4npzw\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-kube-api-access-4npzw\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748587 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748377 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:31.748587 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748432 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:31.748587 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748453 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-token\") pod \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\" (UID: \"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3\") " Jul 30 18:34:31.748587 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748558 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:31.748857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748734 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 18:34:31.748857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748802 2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.748857 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748847 2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-workload-certs\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.749013 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.748860 2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-credential-socket\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.749105 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.749017 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:31.751576 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.751503 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 18:34:31.751767 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.751715 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-kube-api-access-4npzw" (OuterVolumeSpecName: "kube-api-access-4npzw") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "kube-api-access-4npzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:34:31.751901 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.751879 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 18:34:31.751988 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.751969 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" (UID: "c9cc46fa-ff60-4813-9c9f-69a7ced78ff3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 18:34:31.849334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.849299 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-token\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.849334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.849330 2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-data\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.849334 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.849341 2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-podinfo\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.849548 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.849353 2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istiod-ca-cert\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.849548 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.849364 2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-istio-envoy\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.849548 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.849375 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4npzw\" (UniqueName: \"kubernetes.io/projected/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3-kube-api-access-4npzw\") on node \"ip-10-0-140-246.ec2.internal\" DevicePath \"\"" Jul 30 18:34:31.974222 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.974186 2550 generic.go:356] "Generic (PLEG): container finished" podID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerID="cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786" exitCode=0 Jul 30 18:34:31.974411 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.974233 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" Jul 30 18:34:31.974411 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.974283 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" event={"ID":"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3","Type":"ContainerDied","Data":"cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786"} Jul 30 18:34:31.974411 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.974334 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb" event={"ID":"c9cc46fa-ff60-4813-9c9f-69a7ced78ff3","Type":"ContainerDied","Data":"c59d7fb1cccd40cf82d1868d1258696c4eb5187a41243eca6526c3a487d78947"} Jul 30 18:34:31.974411 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.974365 2550 scope.go:117] "RemoveContainer" containerID="cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786" Jul 30 18:34:31.983414 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.983396 2550 scope.go:117] "RemoveContainer" containerID="62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70" Jul 30 18:34:31.992021 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.991997 2550 scope.go:117] "RemoveContainer" containerID="cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786" Jul 30 18:34:31.992277 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:34:31.992260 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786\": container with ID starting with cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786 not found: ID does not exist" containerID="cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786" Jul 30 18:34:31.992346 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.992285 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786"} err="failed to get container status \"cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786\": rpc error: code = NotFound desc = could not find container \"cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786\": container with ID starting with cdae601e26be496005d502d512ffa3ab7c24d4e36b5628b52c0da844a7afc786 not found: ID does not exist" Jul 30 18:34:31.992346 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.992299 2550 scope.go:117] "RemoveContainer" containerID="62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70" Jul 30 18:34:31.992509 ip-10-0-140-246 kubenswrapper[2550]: E0730 18:34:31.992493 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70\": container with ID starting with 62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70 not found: ID does not exist" containerID="62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70" Jul 30 18:34:31.992562 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.992512 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70"} err="failed to get container status \"62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70\": rpc error: code = NotFound desc = could not find container \"62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70\": container with ID starting with 62f4436b9fa63fa285e1941937af5c9844f78a3cdef89a4f53cf29ab34961b70 not found: ID does not exist" Jul 30 18:34:31.993108 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.993043 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb"] Jul 30 18:34:31.996038 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:31.996019 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fbbaa9-openshift-default-d5545fd6d-nxhsb"] Jul 30 18:34:33.807892 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:33.807857 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" path="/var/lib/kubelet/pods/c9cc46fa-ff60-4813-9c9f-69a7ced78ff3/volumes" Jul 30 18:34:33.917649 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:33.917625 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-srl2b_aceb3ec2-9767-473a-bf13-085f5c70815d/copy-manifests/0.log" Jul 30 18:34:33.925319 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:33.925294 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-srl2b_aceb3ec2-9767-473a-bf13-085f5c70815d/manager/0.log" Jul 30 18:34:35.646163 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.646128 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7_e87acff2-b3aa-4e46-a89b-4a48ae2f13d9/pull/0.log" Jul 30 18:34:35.655063 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.655026 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7_e87acff2-b3aa-4e46-a89b-4a48ae2f13d9/extract/0.log" Jul 30 18:34:35.661547 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.661524 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7_e87acff2-b3aa-4e46-a89b-4a48ae2f13d9/util/0.log" Jul 30 18:34:35.802689 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.797106 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m_efdb1c15-eb77-4a88-aa23-9125367e7991/util/0.log" Jul 30 18:34:35.811979 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.811953 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m_efdb1c15-eb77-4a88-aa23-9125367e7991/pull/0.log" Jul 30 18:34:35.834469 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.834442 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m_efdb1c15-eb77-4a88-aa23-9125367e7991/extract/0.log" Jul 30 18:34:35.959076 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.958989 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf_5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc/util/0.log" Jul 30 18:34:35.965952 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.965923 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf_5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc/pull/0.log" Jul 30 18:34:35.973150 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:35.973108 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf_5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc/extract/0.log" Jul 30 18:34:36.094274 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:36.094245 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf_4d495923-6d40-4598-b619-679fd72ff675/util/0.log" Jul 30 18:34:36.101609 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:36.101572 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf_4d495923-6d40-4598-b619-679fd72ff675/pull/0.log" Jul 30 18:34:36.108598 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:36.108573 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf_4d495923-6d40-4598-b619-679fd72ff675/extract/0.log" Jul 30 18:34:37.474021 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:37.473991 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg_3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9/istio-proxy/0.log" Jul 30 18:34:37.602566 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:37.602533 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8qgj9_4dfaea14-d720-462a-9504-602c26734240/discovery/0.log" Jul 30 18:34:37.960715 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:37.960683 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/1.log" Jul 30 18:34:38.088424 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:38.088389 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-s5wsp_3b71deb2-8228-41f5-b81f-af3fde07e727/payload-pre-processing/0.log" Jul 30 18:34:38.330460 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:38.330379 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5447b65c5-h62hv_a5011c56-b6f0-442a-9d09-644fcba552c0/router/0.log" Jul 30 18:34:38.942285 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:38.942252 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-45pp4_d406272b-e819-4a2a-9aff-40e12d0819ed/main/0.log" Jul 30 18:34:39.184611 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:39.184581 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-sqldg_9ce943b5-2bf0-4468-bf83-1fa68a9fc87d/main/0.log" Jul 30 18:34:48.475254 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:48.475212 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8lrfj_246d90ee-d830-4bf0-b942-ffbfa32a98f9/global-pull-secret-syncer/0.log" Jul 30 18:34:48.667792 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:48.667744 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wg4tv_bfd87863-1f53-4966-ba91-7f88ea173688/konnectivity-agent/0.log" Jul 30 18:34:48.747742 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:48.747636 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-246.ec2.internal_2288bbe26fd7af2100c8809ffb7ffbce/haproxy/0.log" Jul 30 18:34:53.113438 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.113404 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7_e87acff2-b3aa-4e46-a89b-4a48ae2f13d9/extract/0.log" Jul 30 18:34:53.152582 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.152548 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7_e87acff2-b3aa-4e46-a89b-4a48ae2f13d9/util/0.log" Jul 30 18:34:53.176704 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.176673 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592v8v7_e87acff2-b3aa-4e46-a89b-4a48ae2f13d9/pull/0.log" Jul 30 18:34:53.205084 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.205048 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m_efdb1c15-eb77-4a88-aa23-9125367e7991/extract/0.log" Jul 30 18:34:53.228668 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.228633 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m_efdb1c15-eb77-4a88-aa23-9125367e7991/util/0.log" Jul 30 18:34:53.252152 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.252124 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jck4m_efdb1c15-eb77-4a88-aa23-9125367e7991/pull/0.log" Jul 30 18:34:53.279664 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.279636 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf_5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc/extract/0.log" Jul 30 18:34:53.310060 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.310031 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf_5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc/util/0.log" Jul 30 18:34:53.332902 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.332861 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p4wgf_5e7b3845-a35e-4eac-ac2c-c9c81b2dbbdc/pull/0.log" Jul 30 18:34:53.360966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.360931 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf_4d495923-6d40-4598-b619-679fd72ff675/extract/0.log" Jul 30 18:34:53.382966 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.382891 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf_4d495923-6d40-4598-b619-679fd72ff675/util/0.log" Jul 30 18:34:53.410458 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:53.410428 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c9nlf_4d495923-6d40-4598-b619-679fd72ff675/pull/0.log" Jul 30 18:34:56.466935 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.466895 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-q4429_ad46dda8-45f4-4f7d-be68-2caf47d7cfa2/kube-state-metrics/0.log" Jul 30 18:34:56.488377 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.488347 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-q4429_ad46dda8-45f4-4f7d-be68-2caf47d7cfa2/kube-rbac-proxy-main/0.log" Jul 30 18:34:56.513566 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.513538 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-q4429_ad46dda8-45f4-4f7d-be68-2caf47d7cfa2/kube-rbac-proxy-self/0.log" Jul 30 18:34:56.710366 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.710335 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bhzcp_fe2178e3-ac80-43c0-8af8-7bd671eef724/node-exporter/0.log" Jul 30 18:34:56.732134 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.732059 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bhzcp_fe2178e3-ac80-43c0-8af8-7bd671eef724/kube-rbac-proxy/0.log" Jul 30 18:34:56.756732 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.756707 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bhzcp_fe2178e3-ac80-43c0-8af8-7bd671eef724/init-textfile/0.log" Jul 30 18:34:56.875037 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.875004 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cbcgw_32532ad3-8620-4c22-b20d-1050a6c50830/kube-rbac-proxy-main/0.log" Jul 30 18:34:56.897921 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.897892 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cbcgw_32532ad3-8620-4c22-b20d-1050a6c50830/kube-rbac-proxy-self/0.log" Jul 30 18:34:56.920951 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:56.920908 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cbcgw_32532ad3-8620-4c22-b20d-1050a6c50830/openshift-state-metrics/0.log" Jul 30 18:34:57.118027 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:57.117996 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-vsnmt_40bbb89d-1774-4199-8915-293a83652819/prometheus-operator/0.log" Jul 30 18:34:57.135838 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:57.135802 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-vsnmt_40bbb89d-1774-4199-8915-293a83652819/kube-rbac-proxy/0.log" Jul 30 18:34:57.162568 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:57.162544 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-hzhhs_85f76932-1052-4cee-b39b-b1eabeed7f84/prometheus-operator-admission-webhook/0.log" Jul 30 18:34:58.312034 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312005 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh"] Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312354 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312365 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312373 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312380 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312385 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" containerName="payload-pre-processing" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312390 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" containerName="payload-pre-processing" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312395 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc963d76-51e3-4abd-b5cd-4b40a9df0c65" containerName="payload-processing" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312401 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc963d76-51e3-4abd-b5cd-4b40a9df0c65" containerName="payload-processing" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312410 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312415 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312420 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e868d194-5090-45a7-9ae9-4ddc9e99bde0" containerName="payload-pre-processing" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312425 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e868d194-5090-45a7-9ae9-4ddc9e99bde0" containerName="payload-pre-processing" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312436 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" Jul 30 18:34:58.312433 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312442 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312448 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312452 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312463 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312468 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312474 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9530180-cfb3-48dd-801b-a3a007b1ec7f" containerName="payload-processing" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312479 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9530180-cfb3-48dd-801b-a3a007b1ec7f" containerName="payload-processing" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312526 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8555d7d-14f9-481d-8ee3-c09b188b0e34" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312534 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312541 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9530180-cfb3-48dd-801b-a3a007b1ec7f" containerName="payload-processing" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312548 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2257821-4879-4e10-9779-b33e21d82b99" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312553 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e868d194-5090-45a7-9ae9-4ddc9e99bde0" containerName="payload-pre-processing" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312559 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9cc46fa-ff60-4813-9c9f-69a7ced78ff3" containerName="istio-proxy" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312564 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc963d76-51e3-4abd-b5cd-4b40a9df0c65" containerName="payload-processing" Jul 30 18:34:58.312920 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.312570 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="af1ce589-bc7d-4d7b-81fc-e1bed390891b" containerName="payload-pre-processing" Jul 30 18:34:58.315535 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.315518 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.318611 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.318590 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp2v8\"/\"kube-root-ca.crt\"" Jul 30 18:34:58.319806 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.319787 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vp2v8\"/\"default-dockercfg-pj59w\"" Jul 30 18:34:58.319929 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.319839 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp2v8\"/\"openshift-service-ca.crt\"" Jul 30 18:34:58.324568 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.324548 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh"] Jul 30 18:34:58.382540 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.382509 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-sys\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.382739 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.382562 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-lib-modules\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.382739 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.382634 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-proc\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.382739 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.382676 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-podres\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.382890 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.382761 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82rvz\" (UniqueName: \"kubernetes.io/projected/a3483980-8607-4ff6-9c21-a2f36cb8eadf-kube-api-access-82rvz\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484107 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484070 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-proc\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484283 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484121 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-podres\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484283 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82rvz\" (UniqueName: \"kubernetes.io/projected/a3483980-8607-4ff6-9c21-a2f36cb8eadf-kube-api-access-82rvz\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484283 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484199 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-proc\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484283 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484207 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-sys\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484283 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484279 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-lib-modules\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484289 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-sys\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484326 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-podres\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.484490 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.484392 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3483980-8607-4ff6-9c21-a2f36cb8eadf-lib-modules\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.493275 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.493256 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82rvz\" (UniqueName: \"kubernetes.io/projected/a3483980-8607-4ff6-9c21-a2f36cb8eadf-kube-api-access-82rvz\") pod \"perf-node-gather-daemonset-hghrh\" (UID: \"a3483980-8607-4ff6-9c21-a2f36cb8eadf\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.581140 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.581027 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-bnbzk_53ddc810-ac04-4499-a5b1-bc820b719759/networking-console-plugin/0.log" Jul 30 18:34:58.626166 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.626140 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:58.763369 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:58.763345 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh"] Jul 30 18:34:58.765780 ip-10-0-140-246 kubenswrapper[2550]: W0730 18:34:58.765753 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda3483980_8607_4ff6_9c21_a2f36cb8eadf.slice/crio-d5d0976a204acd7f253843d547aa94fff3c1a3ebfc2fa58087ada4c3c27248f5 WatchSource:0}: Error finding container d5d0976a204acd7f253843d547aa94fff3c1a3ebfc2fa58087ada4c3c27248f5: Status 404 returned error can't find the container with id d5d0976a204acd7f253843d547aa94fff3c1a3ebfc2fa58087ada4c3c27248f5 Jul 30 18:34:59.115587 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:59.115547 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" event={"ID":"a3483980-8607-4ff6-9c21-a2f36cb8eadf","Type":"ContainerStarted","Data":"0529657d3640f9ea96d929b623fe219548706a3fbd12e791eb5fc8947030d9ea"} Jul 30 18:34:59.115772 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:59.115602 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:34:59.115772 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:59.115618 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" event={"ID":"a3483980-8607-4ff6-9c21-a2f36cb8eadf","Type":"ContainerStarted","Data":"d5d0976a204acd7f253843d547aa94fff3c1a3ebfc2fa58087ada4c3c27248f5"} Jul 30 18:34:59.131725 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:59.131676 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" podStartSLOduration=1.13166086 podStartE2EDuration="1.13166086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 18:34:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:34:59.131304268 +0000 UTC m=+4093.863818953" watchObservedRunningTime="2026-07-30 18:34:59.13166086 +0000 UTC m=+4093.864175605" Jul 30 18:34:59.816502 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:34:59.816476 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-gsj92_ae5247db-98b7-47a7-bc64-ab7db630ee16/download-server/0.log" Jul 30 18:35:01.565897 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:01.565864 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb9js_0df4a595-c599-46ba-aa60-d2dc6bd50806/dns/0.log" Jul 30 18:35:01.588349 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:01.588307 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb9js_0df4a595-c599-46ba-aa60-d2dc6bd50806/kube-rbac-proxy/0.log" Jul 30 18:35:01.640904 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:01.640873 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mptvx_0107a065-c0b7-496d-ad29-8d02b454374c/dns-node-resolver/0.log" Jul 30 18:35:02.259780 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:02.259746 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8hj5m_ceb77fd3-c224-4c26-aad4-923719f14703/node-ca/0.log" Jul 30 18:35:03.272610 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:03.272580 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg_3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9/istio-proxy/0.log" Jul 30 18:35:03.550105 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:03.550015 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8qgj9_4dfaea14-d720-462a-9504-602c26734240/discovery/0.log" Jul 30 18:35:03.635323 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:03.635294 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/0.log" Jul 30 18:35:03.696345 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:03.696302 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/1.log" Jul 30 18:35:03.737414 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:03.737383 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-s5wsp_3b71deb2-8228-41f5-b81f-af3fde07e727/payload-pre-processing/0.log" Jul 30 18:35:03.806653 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:03.806559 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5447b65c5-h62hv_a5011c56-b6f0-442a-9d09-644fcba552c0/router/0.log" Jul 30 18:35:04.626672 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:04.626643 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5g9cd_1743b5d7-6896-4df5-9535-49be6477dc1d/serve-healthcheck-canary/0.log" Jul 30 18:35:05.131489 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:05.131462 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-hghrh" Jul 30 18:35:05.261448 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:05.261423 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jqgd7_79c050c9-18f5-4538-b6b0-adc4224c24c9/kube-rbac-proxy/0.log" Jul 30 18:35:05.283747 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:05.283720 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jqgd7_79c050c9-18f5-4538-b6b0-adc4224c24c9/exporter/0.log" Jul 30 18:35:05.308806 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:05.308780 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jqgd7_79c050c9-18f5-4538-b6b0-adc4224c24c9/extractor/0.log" Jul 30 18:35:08.584126 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:08.584098 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-srl2b_aceb3ec2-9767-473a-bf13-085f5c70815d/manager/0.log" Jul 30 18:35:08.608921 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:08.608886 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-srl2b_aceb3ec2-9767-473a-bf13-085f5c70815d/copy-manifests/0.log" Jul 30 18:35:09.200343 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.200308 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg_3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9/istio-proxy/0.log" Jul 30 18:35:09.477174 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.477053 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8qgj9_4dfaea14-d720-462a-9504-602c26734240/discovery/0.log" Jul 30 18:35:09.529526 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.529486 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/0.log" Jul 30 18:35:09.582126 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.582088 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/1.log" Jul 30 18:35:09.612091 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.612057 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-s5wsp_3b71deb2-8228-41f5-b81f-af3fde07e727/payload-pre-processing/0.log" Jul 30 18:35:09.661043 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.661005 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5447b65c5-h62hv_a5011c56-b6f0-442a-9d09-644fcba552c0/router/0.log" Jul 30 18:35:09.882025 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:09.881992 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5cdfbb974d-mvjbr_60ad5ce0-3621-48a5-a324-65bb968b8589/manager/0.log" Jul 30 18:35:11.052457 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:11.052423 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7stwg_3dab0ff8-c7ef-4ad3-bf11-91e10f6774d9/istio-proxy/0.log" Jul 30 18:35:11.327244 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:11.327163 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8qgj9_4dfaea14-d720-462a-9504-602c26734240/discovery/0.log" Jul 30 18:35:11.384432 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:11.384389 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/0.log" Jul 30 18:35:11.441675 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:11.441644 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9s4rp_910ebae8-59cf-41b7-b3d0-0099badb3ecf/istio-proxy/1.log" Jul 30 18:35:11.482578 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:11.482549 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-s5wsp_3b71deb2-8228-41f5-b81f-af3fde07e727/payload-pre-processing/0.log" Jul 30 18:35:11.525470 ip-10-0-140-246 kubenswrapper[2550]: I0730 18:35:11.525441 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5447b65c5-h62hv_a5011c56-b6f0-442a-9d09-644fcba552c0/router/0.log"