Jul 22 11:47:32.247958 ip-10-0-143-104 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 22 11:47:32.247970 ip-10-0-143-104 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 22 11:47:32.247977 ip-10-0-143-104 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 22 11:47:32.248195 ip-10-0-143-104 systemd[1]: Failed to start Kubernetes Kubelet. Jul 22 11:47:42.253376 ip-10-0-143-104 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 22 11:47:42.253423 ip-10-0-143-104 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cd3d90af1841493d8739848102bb9502 -- Jul 22 11:49:31.571104 ip-10-0-143-104 systemd[1]: Starting Kubernetes Kubelet... Jul 22 11:49:31.910423 ip-10-0-143-104 kubenswrapper[2534]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 11:49:31.910423 ip-10-0-143-104 kubenswrapper[2534]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 11:49:31.910423 ip-10-0-143-104 kubenswrapper[2534]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 11:49:31.910423 ip-10-0-143-104 kubenswrapper[2534]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 11:49:31.910423 ip-10-0-143-104 kubenswrapper[2534]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 11:49:31.911984 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.911895 2534 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 11:49:31.916066 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916050 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:49:31.916066 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916066 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916070 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916073 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916077 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916080 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916083 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916086 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916088 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916091 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916094 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916096 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916099 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916102 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916104 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916112 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916116 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916120 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916123 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916126 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916129 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:49:31.916128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916132 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916134 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916137 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916140 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916142 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916145 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916148 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916150 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916153 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916156 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916158 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916161 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916163 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916166 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916168 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916171 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916173 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916175 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916179 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:49:31.916591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916183 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916186 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916189 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916192 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916195 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916197 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916200 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916203 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916206 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916208 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916210 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916213 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916215 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916217 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916220 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916223 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916226 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916228 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916230 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916233 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:49:31.917053 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916235 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916238 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916240 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916243 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916245 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916248 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916250 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916253 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916255 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916257 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916260 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916263 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916265 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916268 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916270 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916273 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916276 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916278 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916280 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916283 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:49:31.917536 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916285 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916288 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916290 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916295 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916298 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916301 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916305 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916308 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916311 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916314 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916317 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916319 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916321 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916324 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916326 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916328 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916331 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916333 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.916336 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:49:31.918060 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917582 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917590 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917593 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917596 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917600 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917604 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917607 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917609 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917612 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917616 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917618 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917621 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917623 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917626 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917629 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917632 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917634 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917637 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917640 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:49:31.918896 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917642 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917647 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917650 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917652 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917655 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917657 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917660 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917662 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917665 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917667 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917670 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917672 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917675 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917677 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917679 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917682 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917684 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917687 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917689 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917691 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:49:31.919589 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917694 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917696 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917699 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917701 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917703 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917706 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917708 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917711 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917713 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917716 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917718 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917721 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917723 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917726 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917729 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917731 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917734 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917736 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917739 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917741 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:49:31.920133 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917743 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917746 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917750 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917753 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917756 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917759 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917761 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917764 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917766 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917768 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917771 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917773 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917776 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917778 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917780 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917783 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917785 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917788 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917803 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917806 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:49:31.920629 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917808 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917811 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917814 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917818 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917822 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917825 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917828 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917831 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917833 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917835 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917838 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917840 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917843 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917845 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917848 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917850 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917853 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917855 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917858 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:49:31.921119 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.917860 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920388 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920401 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920404 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920407 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920410 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920413 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920417 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920420 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920422 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920425 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920428 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920432 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920435 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920439 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920443 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920447 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920449 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:49:31.921712 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920452 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920457 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920460 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920468 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920471 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920474 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920477 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920480 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920483 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920486 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920489 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920492 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920495 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920497 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920500 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920503 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920506 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920508 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920511 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:49:31.922172 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920513 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920516 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920518 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920526 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920529 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920531 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920534 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920536 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920539 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920541 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920544 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920546 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920549 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920551 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920554 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920557 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920559 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920562 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920564 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920567 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:49:31.922649 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920569 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920572 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920574 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920576 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920579 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920582 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920584 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920586 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920589 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920591 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920594 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920596 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920599 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920601 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920604 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920607 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920609 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920611 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920614 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920616 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:49:31.923145 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920619 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920621 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920623 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920626 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920628 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920631 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920634 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920638 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920640 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920643 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920645 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920648 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920651 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920654 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920656 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920659 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920661 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920664 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920666 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920669 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:49:31.923614 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920671 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920674 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.920676 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920750 2534 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920758 2534 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920764 2534 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920768 2534 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920772 2534 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920775 2534 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920780 2534 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920784 2534 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920787 2534 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920810 2534 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920816 2534 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920821 2534 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920826 2534 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920830 2534 flags.go:64] FLAG: --cgroup-root="" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920833 2534 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920837 2534 flags.go:64] FLAG: --client-ca-file="" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920839 2534 flags.go:64] FLAG: --cloud-provider="external" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920844 2534 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920849 2534 flags.go:64] FLAG: --cluster-domain="" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920852 2534 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920855 2534 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 11:49:31.924127 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920859 2534 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920862 2534 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920866 2534 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920869 2534 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920873 2534 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920876 2534 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920879 2534 flags.go:64] FLAG: --contention-profiling="false" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920882 2534 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920885 2534 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920888 2534 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920891 2534 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920900 2534 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920903 2534 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920906 2534 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920909 2534 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920912 2534 flags.go:64] FLAG: --enable-server="true" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920915 2534 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920919 2534 flags.go:64] FLAG: --event-burst="100" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920922 2534 flags.go:64] FLAG: --event-qps="50" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920925 2534 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920927 2534 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920930 2534 flags.go:64] FLAG: --eviction-hard="" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920934 2534 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920936 2534 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920939 2534 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 11:49:31.924719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920942 2534 flags.go:64] FLAG: --eviction-soft="" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920945 2534 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920948 2534 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920951 2534 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920955 2534 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920957 2534 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920960 2534 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920963 2534 flags.go:64] FLAG: --feature-gates="" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920967 2534 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920970 2534 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920973 2534 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920976 2534 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920979 2534 flags.go:64] FLAG: --healthz-port="10248" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920981 2534 flags.go:64] FLAG: --help="false" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920984 2534 flags.go:64] FLAG: --hostname-override="ip-10-0-143-104.ec2.internal" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920987 2534 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920990 2534 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920993 2534 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.920996 2534 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921000 2534 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921002 2534 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921005 2534 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921008 2534 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 11:49:31.925328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921010 2534 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921013 2534 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921016 2534 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921019 2534 flags.go:64] FLAG: --kube-reserved="" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921022 2534 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921025 2534 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921028 2534 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921030 2534 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921033 2534 flags.go:64] FLAG: --lock-file="" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921035 2534 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921038 2534 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921041 2534 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921047 2534 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921050 2534 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921054 2534 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921057 2534 flags.go:64] FLAG: --logging-format="text" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921060 2534 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921063 2534 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921066 2534 flags.go:64] FLAG: --manifest-url="" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921069 2534 flags.go:64] FLAG: --manifest-url-header="" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921078 2534 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921081 2534 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921085 2534 flags.go:64] FLAG: --max-pods="110" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921088 2534 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921091 2534 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 11:49:31.925925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921094 2534 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921096 2534 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921099 2534 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921102 2534 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921105 2534 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921113 2534 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921116 2534 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921118 2534 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921121 2534 flags.go:64] FLAG: --pod-cidr="" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921124 2534 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921130 2534 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921133 2534 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921137 2534 flags.go:64] FLAG: --pods-per-core="0" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921139 2534 flags.go:64] FLAG: --port="10250" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921142 2534 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921145 2534 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08d0434ab2c086fa9" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921148 2534 flags.go:64] FLAG: --qos-reserved="" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921151 2534 flags.go:64] FLAG: --read-only-port="10255" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921156 2534 flags.go:64] FLAG: --register-node="true" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921159 2534 flags.go:64] FLAG: --register-with-taints="" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921162 2534 flags.go:64] FLAG: --registry-burst="10" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921165 2534 flags.go:64] FLAG: --registry-qps="5" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921168 2534 flags.go:64] FLAG: --reserved-cpus="" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921171 2534 flags.go:64] FLAG: --reserved-memory="" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921175 2534 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 11:49:31.926520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921178 2534 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921180 2534 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921183 2534 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921186 2534 flags.go:64] FLAG: --runonce="false" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921189 2534 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921192 2534 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921195 2534 flags.go:64] FLAG: --seccomp-default="false" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921198 2534 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921201 2534 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921203 2534 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921206 2534 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921209 2534 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921212 2534 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921214 2534 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921217 2534 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921220 2534 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921223 2534 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921226 2534 flags.go:64] FLAG: --system-cgroups="" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921228 2534 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921233 2534 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921236 2534 flags.go:64] FLAG: --tls-cert-file="" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921238 2534 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921243 2534 flags.go:64] FLAG: --tls-min-version="" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921246 2534 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921248 2534 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 11:49:31.927151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921252 2534 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921255 2534 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921258 2534 flags.go:64] FLAG: --v="2" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921261 2534 flags.go:64] FLAG: --version="false" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921266 2534 flags.go:64] FLAG: --vmodule="" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921270 2534 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.921273 2534 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921397 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921401 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921404 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921407 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921409 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921412 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921414 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921417 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921420 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921422 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921424 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921427 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921429 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:49:31.927905 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921432 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921445 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921448 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921452 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921456 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921459 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921462 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921464 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921466 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921469 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921472 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921474 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921478 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921481 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921484 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921486 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921489 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921491 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921496 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921499 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:49:31.928773 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921502 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921504 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921507 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921509 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921512 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921514 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921517 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921519 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921522 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921524 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921526 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921529 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921531 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921534 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921536 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921538 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921541 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921543 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921546 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921548 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:49:31.929645 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921551 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921554 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921556 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921559 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921563 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921567 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921571 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921574 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921577 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921579 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921583 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921586 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921589 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921591 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921593 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921596 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921598 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921601 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921605 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:49:31.930509 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921608 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921611 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921613 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921616 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921618 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921621 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921623 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921625 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921628 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921630 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921633 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921635 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921638 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921640 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921643 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921645 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921647 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921651 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921653 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:49:31.931354 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921656 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921658 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921661 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921663 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921667 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921670 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921673 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.921675 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.922268 2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.930096 2534 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.930121 2534 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.930148 2534 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.930158 2534 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930248 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930259 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:49:31.932103 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930266 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930271 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930276 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930281 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930285 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930290 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930294 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930298 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930303 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930307 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930311 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930315 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930320 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930324 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930328 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930332 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930336 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930340 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930344 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930349 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:49:31.932725 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930353 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930357 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930361 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930365 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930370 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930375 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930379 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930385 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930389 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930394 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930398 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930402 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930406 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930411 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930415 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930420 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930424 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930428 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930433 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930438 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:49:31.933591 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930442 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930446 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930450 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930454 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930459 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930463 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930467 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930471 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930475 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930479 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930483 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930487 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930491 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930495 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930499 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930503 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930507 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930538 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930545 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930551 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:49:31.934128 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930557 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930561 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930565 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930569 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930573 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930578 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930582 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930586 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930590 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930595 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930599 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930603 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930607 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930611 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930615 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930619 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930623 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930628 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930631 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930636 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:49:31.934762 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930640 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930644 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930648 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930652 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930656 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930660 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930664 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930668 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930672 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930676 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930684 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930691 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930697 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930703 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930707 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930712 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:49:31.935290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930717 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.930725 2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930931 2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930940 2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930945 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930950 2534 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930955 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930959 2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930964 2534 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930968 2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930973 2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930977 2534 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930981 2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930985 2534 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930991 2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:49:31.935736 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.930997 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931002 2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931006 2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931010 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931014 2534 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931018 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931022 2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931026 2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931031 2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931035 2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931039 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931043 2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931048 2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931052 2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931056 2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931061 2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931066 2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931070 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931074 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931078 2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:49:31.936262 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931083 2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931089 2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931094 2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931099 2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931104 2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931108 2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931114 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931118 2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931122 2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931126 2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931130 2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931135 2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931140 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931144 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931150 2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931155 2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931159 2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931164 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931168 2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:49:31.936776 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931172 2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931176 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931180 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931184 2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931188 2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931193 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931198 2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931203 2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931207 2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931212 2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931216 2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931223 2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931228 2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931233 2534 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931237 2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931242 2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931246 2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931250 2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931254 2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:49:31.937284 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931258 2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931263 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931267 2534 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931271 2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931275 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931280 2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931284 2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931288 2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931292 2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931296 2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931300 2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931304 2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931308 2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931312 2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931316 2534 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931320 2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931324 2534 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931329 2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931333 2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931337 2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:49:31.938014 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931341 2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931345 2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931349 2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931354 2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931358 2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931362 2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931365 2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:31.931370 2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.931377 2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.931568 2534 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.934269 2534 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.935563 2534 server.go:1019] "Starting client certificate rotation" Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.935664 2534 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 11:49:31.938863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.935735 2534 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 11:49:31.939444 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.938151 2534 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 11:49:31.939891 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.939872 2534 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 11:49:31.949499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.949478 2534 log.go:25] "Validated CRI v1 runtime API" Jul 22 11:49:31.953385 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.953368 2534 log.go:25] "Validated CRI v1 image API" Jul 22 11:49:31.954112 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.954093 2534 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 11:49:31.956322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.956304 2534 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a4e09f23-e06a-4fc7-9192-c5f4f27fbd16:/dev/nvme0n1p4 e9a860d1-2b46-4742-98af-b1030316d7f8:/dev/nvme0n1p3] Jul 22 11:49:31.956379 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.956323 2534 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 11:49:31.962461 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.962314 2534 manager.go:216] Machine: {Timestamp:2026-07-22 11:49:31.961153218 +0000 UTC m=+0.287000472 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3075219 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24cf3ffb94b441417db1aa960d967c SystemUUID:ec24cf3f-fb94-b441-417d-b1aa960d967c BootID:cd3d90af-1841-493d-8739-848102bb9502 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4e:c1:65:3a:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4e:c1:65:3a:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:b1:5f:30:0e:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 11:49:31.962461 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.962457 2534 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 11:49:31.962565 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.962545 2534 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 11:49:31.964914 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.964887 2534 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 11:49:31.965081 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.964919 2534 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 11:49:31.965586 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.965574 2534 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 11:49:31.965616 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.965588 2534 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 11:49:31.965616 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.965595 2534 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 11:49:31.966206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.966193 2534 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 11:49:31.967563 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.967550 2534 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 11:49:31.971590 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.971574 2534 state_mem.go:36] "Initialized new in-memory state store" Jul 22 11:49:31.971738 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.971728 2534 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 11:49:31.972343 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972334 2534 kubelet.go:486] "Attempting to sync node with API server" Jul 22 11:49:31.972374 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972358 2534 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 11:49:31.972400 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972378 2534 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 11:49:31.972400 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972391 2534 kubelet.go:398] "Adding apiserver pod source" Jul 22 11:49:31.972462 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972433 2534 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 11:49:31.972825 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972814 2534 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 11:49:31.972857 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.972833 2534 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 11:49:31.974913 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.974887 2534 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 11:49:31.976718 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.976701 2534 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 11:49:31.976836 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.976741 2534 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 11:49:31.978286 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978270 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 11:49:31.978336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978293 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 11:49:31.978336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978305 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 11:49:31.978336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978314 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 11:49:31.978336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978325 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 11:49:31.978336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978334 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 11:49:31.978515 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978346 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 11:49:31.978515 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978355 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 11:49:31.978515 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978365 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 11:49:31.978515 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978371 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 11:49:31.978515 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978380 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 11:49:31.978515 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.978385 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 11:49:31.979542 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.979525 2534 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 11:49:31.979582 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.979565 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 11:49:31.979582 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.979573 2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 11:49:31.982922 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.982906 2534 server.go:1268] "Started kubelet" Jul 22 11:49:31.983038 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.983013 2534 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 11:49:31.983091 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.983004 2534 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 11:49:31.983091 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.983080 2534 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 11:49:31.985787 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.985689 2534 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 11:49:31.985787 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:31.985696 2534 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 11:49:31.985787 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:31.985773 2534 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 11:49:31.985775 ip-10-0-143-104 systemd[1]: Started Kubernetes Kubelet. Jul 22 11:49:31.987038 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.986677 2534 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 11:49:31.987987 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.987970 2534 server.go:310] "Adding debug handlers to kubelet server" Jul 22 11:49:31.989309 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:31.989286 2534 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 11:49:31.994053 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:31.993190 2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-104.ec2.internal.18c49a44a63aab7b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-104.ec2.internal,UID:ip-10-0-143-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-104.ec2.internal,},FirstTimestamp:2026-07-22 11:49:31.982875515 +0000 UTC m=+0.308722770,LastTimestamp:2026-07-22 11:49:31.982875515 +0000 UTC m=+0.308722770,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-104.ec2.internal,}" Jul 22 11:49:31.994581 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.994563 2534 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 11:49:31.994687 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.994584 2534 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 11:49:31.994734 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:31.994705 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:31.995141 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995121 2534 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 11:49:31.995239 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995156 2534 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 11:49:31.995239 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995174 2534 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 11:49:31.995353 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995339 2534 reconstruct.go:97] "Volume reconstruction finished" Jul 22 11:49:31.995353 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995352 2534 reconciler.go:29] "Reconciler: start to sync state" Jul 22 11:49:31.995453 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995410 2534 factory.go:55] Registering systemd factory Jul 22 11:49:31.995453 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995428 2534 factory.go:223] Registration of the systemd container factory successfully Jul 22 11:49:31.995645 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995627 2534 factory.go:153] Registering CRI-O factory Jul 22 11:49:31.995645 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995646 2534 factory.go:223] Registration of the crio container factory successfully Jul 22 11:49:31.995815 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995700 2534 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 11:49:31.995815 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995726 2534 factory.go:103] Registering Raw factory Jul 22 11:49:31.995815 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.995741 2534 manager.go:1196] Started watching for new ooms in manager Jul 22 11:49:31.996147 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:31.996133 2534 manager.go:349] Starting recovery of all containers Jul 22 11:49:32.006409 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.006264 2534 manager.go:354] Recovery completed Jul 22 11:49:32.010390 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.010362 2534 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 11:49:32.010759 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.010746 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.010896 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.010875 2534 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 11:49:32.013972 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.013947 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.014048 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.013984 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.014048 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.013998 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.015075 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015060 2534 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 11:49:32.015075 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015072 2534 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 11:49:32.015170 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015087 2534 state_mem.go:36] "Initialized new in-memory state store" Jul 22 11:49:32.015170 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015128 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.015834 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015821 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.015882 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015850 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.015882 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.015868 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.017253 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.017237 2534 policy_none.go:49] "None policy: Start" Jul 22 11:49:32.017253 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.017255 2534 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 11:49:32.017345 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.017266 2534 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 11:49:32.019226 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.019215 2534 policy_none.go:47] "Start" Jul 22 11:49:32.030856 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.030756 2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-104.ec2.internal.18c49a44a81527de default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-104.ec2.internal,UID:ip-10-0-143-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-104.ec2.internal,},FirstTimestamp:2026-07-22 11:49:32.013971422 +0000 UTC m=+0.339818683,LastTimestamp:2026-07-22 11:49:32.013971422 +0000 UTC m=+0.339818683,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-104.ec2.internal,}" Jul 22 11:49:32.047364 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.047344 2534 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nllf8" Jul 22 11:49:32.048715 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.048634 2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-104.ec2.internal.18c49a44a8156f8b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-104.ec2.internal,UID:ip-10-0-143-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-104.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-104.ec2.internal,},FirstTimestamp:2026-07-22 11:49:32.013989771 +0000 UTC m=+0.339837026,LastTimestamp:2026-07-22 11:49:32.013989771 +0000 UTC m=+0.339837026,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-104.ec2.internal,}" Jul 22 11:49:32.062009 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.061932 2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-104.ec2.internal.18c49a44a815a391 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-104.ec2.internal,UID:ip-10-0-143-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-104.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-104.ec2.internal,},FirstTimestamp:2026-07-22 11:49:32.014003089 +0000 UTC m=+0.339850346,LastTimestamp:2026-07-22 11:49:32.014003089 +0000 UTC m=+0.339850346,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-104.ec2.internal,}" Jul 22 11:49:32.062211 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.062193 2534 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nllf8" Jul 22 11:49:32.063612 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.063591 2534 manager.go:337] "Starting Device Plugin manager" Jul 22 11:49:32.063733 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.063638 2534 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 11:49:32.063733 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.063664 2534 server.go:87] "Starting device plugin registration server" Jul 22 11:49:32.064005 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.063990 2534 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 11:49:32.064081 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.064008 2534 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 11:49:32.064128 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.064093 2534 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 11:49:32.064179 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.064166 2534 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 11:49:32.064216 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.064181 2534 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 11:49:32.065002 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.064983 2534 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 11:49:32.065085 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.065028 2534 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.093366 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.093320 2534 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 11:49:32.095506 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.094713 2534 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 11:49:32.095506 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.094752 2534 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 11:49:32.095506 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.094773 2534 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 11:49:32.095506 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.094828 2534 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 11:49:32.097743 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.097724 2534 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:49:32.164284 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.164150 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.165389 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.165373 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.165470 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.165409 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.165470 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.165427 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.165470 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.165462 2534 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.176694 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.176672 2534 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.176757 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.176697 2534 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-104.ec2.internal\": node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.195046 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.195003 2534 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal"] Jul 22 11:49:32.195174 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.195082 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.195492 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.195475 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1826b170d96cfa607cb3c14f722f823e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal\" (UID: \"1826b170d96cfa607cb3c14f722f823e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.195551 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.195499 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1826b170d96cfa607cb3c14f722f823e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal\" (UID: \"1826b170d96cfa607cb3c14f722f823e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.196500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.196482 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.196555 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.196510 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.196555 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.196520 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.198869 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.198857 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.199040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199025 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.199081 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199059 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.199571 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199545 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.199571 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199575 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.199719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199585 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.199719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199545 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.199719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199662 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.199719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.199680 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.201913 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.201898 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.201913 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.201926 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:32.202129 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.202110 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.202580 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.202562 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:32.202663 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.202585 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:32.202663 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.202599 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:32.226482 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.226451 2534 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-104.ec2.internal\" not found" node="ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.229931 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.229914 2534 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-104.ec2.internal\" not found" node="ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.295616 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.295577 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1252a5c274066b1c97fa5a755cd7d35-config\") pod \"kube-apiserver-proxy-ip-10-0-143-104.ec2.internal\" (UID: \"a1252a5c274066b1c97fa5a755cd7d35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.295754 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.295637 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1826b170d96cfa607cb3c14f722f823e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal\" (UID: \"1826b170d96cfa607cb3c14f722f823e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.295754 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.295656 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1826b170d96cfa607cb3c14f722f823e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal\" (UID: \"1826b170d96cfa607cb3c14f722f823e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.295754 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.295703 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1826b170d96cfa607cb3c14f722f823e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal\" (UID: \"1826b170d96cfa607cb3c14f722f823e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.295754 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.295703 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1826b170d96cfa607cb3c14f722f823e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal\" (UID: \"1826b170d96cfa607cb3c14f722f823e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.302736 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.302715 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.396671 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.396627 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1252a5c274066b1c97fa5a755cd7d35-config\") pod \"kube-apiserver-proxy-ip-10-0-143-104.ec2.internal\" (UID: \"a1252a5c274066b1c97fa5a755cd7d35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.396830 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.396702 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1252a5c274066b1c97fa5a755cd7d35-config\") pod \"kube-apiserver-proxy-ip-10-0-143-104.ec2.internal\" (UID: \"a1252a5c274066b1c97fa5a755cd7d35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.403806 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.403765 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.504617 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.504544 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.529744 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.529720 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.532890 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.532870 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" Jul 22 11:49:32.604814 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.604767 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.705351 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.705314 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.805926 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.805839 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.862688 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.862658 2534 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:49:32.906157 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:32.906123 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:32.935309 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.935278 2534 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 11:49:32.935917 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.935453 2534 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 11:49:32.935917 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.935480 2534 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 11:49:32.999418 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:32.999385 2534 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:49:33.006663 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.006635 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:33.069661 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.069585 2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 11:44:33 +0000 UTC" deadline="2028-01-18 17:11:22.529664879 +0000 UTC" Jul 22 11:49:33.069661 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.069623 2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13085h21m49.460045453s" Jul 22 11:49:33.087762 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.087736 2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:49:33.088938 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.088919 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:49:33.089034 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.088951 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:49:33.089034 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.088963 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:49:33.102049 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:33.102024 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1826b170d96cfa607cb3c14f722f823e.slice/crio-aefaa0f728cb7c9cf692b20622c6480af3e1fd5d64bbac5c7bcdd7a590a00acc WatchSource:0}: Error finding container aefaa0f728cb7c9cf692b20622c6480af3e1fd5d64bbac5c7bcdd7a590a00acc: Status 404 returned error can't find the container with id aefaa0f728cb7c9cf692b20622c6480af3e1fd5d64bbac5c7bcdd7a590a00acc Jul 22 11:49:33.102528 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:33.102512 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1252a5c274066b1c97fa5a755cd7d35.slice/crio-70cce3203c29f1480504a71d486304d3c6ee4b19e25c4e6d52c56ed2e25e3fd5 WatchSource:0}: Error finding container 70cce3203c29f1480504a71d486304d3c6ee4b19e25c4e6d52c56ed2e25e3fd5: Status 404 returned error can't find the container with id 70cce3203c29f1480504a71d486304d3c6ee4b19e25c4e6d52c56ed2e25e3fd5 Jul 22 11:49:33.106843 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.106824 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:33.106843 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.106837 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 11:49:33.207628 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.207575 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:33.308190 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.308151 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:33.409133 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.409041 2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-104.ec2.internal\" not found" Jul 22 11:49:33.453531 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.453504 2534 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:49:33.495530 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.495493 2534 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" Jul 22 11:49:33.512899 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.512869 2534 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 11:49:33.513889 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.513856 2534 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" Jul 22 11:49:33.525040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.525015 2534 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 11:49:33.974752 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.974484 2534 apiserver.go:52] "Watching apiserver" Jul 22 11:49:33.983283 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.983093 2534 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 11:49:33.984204 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.984134 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-q2zv9","openshift-dns/node-resolver-ngkzd","openshift-multus/multus-8cdj6","openshift-multus/network-metrics-daemon-sw68n","openshift-network-diagnostics/network-check-target-9gbj2","openshift-network-operator/iptables-alerter-6cwcg","openshift-ovn-kubernetes/ovnkube-node-2g487","kube-system/global-pull-secret-syncer-dkgxt","kube-system/konnectivity-agent-b8zxh","kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm","openshift-cluster-node-tuning-operator/tuned-qmsn4","openshift-image-registry/node-ca-4c6jg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal"] Jul 22 11:49:33.987538 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.987513 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.990398 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.990574 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.990797 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.991090 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.991149 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.991093 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.991192 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tkf6l\"" Jul 22 11:49:33.991321 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.991097 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 11:49:33.994312 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.994292 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-czj6w\"" Jul 22 11:49:33.994676 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.994653 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 11:49:33.994864 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.994847 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 11:49:33.995097 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.995080 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 11:49:33.995315 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.995296 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8cdj6" Jul 22 11:49:33.995823 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.995803 2534 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 11:49:33.996943 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.996922 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:33.997068 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.997023 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:33.998875 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:33.998853 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:33.999560 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:33.999534 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:34.000241 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.000221 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 11:49:34.000475 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.000458 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 11:49:34.000752 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.000738 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 11:49:34.000985 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.000967 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zxp6h\"" Jul 22 11:49:34.001090 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.001069 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 11:49:34.001408 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.001391 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.005121 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.004883 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.005224 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005165 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-cni-multus\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005224 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005216 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-systemd\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005242 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-cni-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005266 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njnpb\" (UniqueName: \"kubernetes.io/projected/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-kube-api-access-njnpb\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005295 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-log-socket\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005342 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovnkube-config\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005365 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-cni-bin\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005392 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt5ct\" (UniqueName: \"kubernetes.io/projected/80dd74e9-d654-4fa1-98bf-2bcf2395da42-kube-api-access-mt5ct\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:34.005482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005417 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-node-log\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005442 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005468 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfqcb\" (UniqueName: \"kubernetes.io/projected/ca0eef2d-7095-4da1-84cc-c0d803143be7-kube-api-access-sfqcb\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005491 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-cni-binary-copy\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005518 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-etc-kubernetes\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005542 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-kubelet\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005565 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-system-cni-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005588 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-daemon-config\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005611 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005642 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-run-netns\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005665 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-ovn\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005688 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-cni-bin\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005713 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-cni-netd\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005736 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovn-node-metrics-cert\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005761 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-os-release\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.005799 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005785 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-multus-certs\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005811 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-etc-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005844 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovnkube-script-lib\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005900 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-socket-dir-parent\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005929 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-netns\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005956 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-hostroot\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.005986 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-conf-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006009 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-systemd-units\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006034 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006059 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006087 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fdb909c-8893-43f5-ba58-a8f774076194-host\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006132 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7mhj\" (UniqueName: \"kubernetes.io/projected/1fdb909c-8893-43f5-ba58-a8f774076194-kube-api-access-v7mhj\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006161 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-cnibin\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006186 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-kubelet\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006212 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006237 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-slash\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006263 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-var-lib-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.006543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006288 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-env-overrides\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.007292 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006311 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fdb909c-8893-43f5-ba58-a8f774076194-serviceca\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.007292 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.006335 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-k8s-cni-cncf-io\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.007292 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.007009 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jxqpl\"" Jul 22 11:49:34.007292 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.007013 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:49:34.007292 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.007159 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 11:49:34.007292 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.007237 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 11:49:34.007512 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.007443 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.008959 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.008935 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 11:49:34.009336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.009313 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7x88j\"" Jul 22 11:49:34.009534 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.009517 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 11:49:34.010009 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.009991 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 11:49:34.010097 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.010066 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6lml9\"" Jul 22 11:49:34.010177 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.010155 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.010293 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.010274 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 11:49:34.012675 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.012526 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 11:49:34.016102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.012865 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 11:49:34.016102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.013104 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qm6jt\"" Jul 22 11:49:34.016102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.013239 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.016102 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.013337 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:34.016102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.014351 2534 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 11:49:34.016102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.015392 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.018207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.017729 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 11:49:34.018207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.017988 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x4x8z\"" Jul 22 11:49:34.018358 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.018262 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.018358 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.018334 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 11:49:34.018531 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.018267 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 11:49:34.020626 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.020603 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:49:34.020986 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.020967 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rc6t5\"" Jul 22 11:49:34.021230 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.021212 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 11:49:34.034813 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.034789 2534 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-58p5h" Jul 22 11:49:34.041538 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.041512 2534 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-58p5h" Jul 22 11:49:34.096809 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.096776 2534 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 11:49:34.100809 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.100661 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" event={"ID":"a1252a5c274066b1c97fa5a755cd7d35","Type":"ContainerStarted","Data":"70cce3203c29f1480504a71d486304d3c6ee4b19e25c4e6d52c56ed2e25e3fd5"} Jul 22 11:49:34.102980 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.102951 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" event={"ID":"1826b170d96cfa607cb3c14f722f823e","Type":"ContainerStarted","Data":"aefaa0f728cb7c9cf692b20622c6480af3e1fd5d64bbac5c7bcdd7a590a00acc"} Jul 22 11:49:34.106653 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106626 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-tuned\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.106783 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106670 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f2645b-2fc8-4bef-9bec-128198246ad6-tmp-dir\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.106783 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106707 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovnkube-config\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.106783 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106736 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt5ct\" (UniqueName: \"kubernetes.io/projected/80dd74e9-d654-4fa1-98bf-2bcf2395da42-kube-api-access-mt5ct\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:34.106783 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106762 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-lib-modules\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106786 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83528e59-ca1b-4ae2-b520-6fd11121829e-agent-certs\") pod \"konnectivity-agent-b8zxh\" (UID: \"83528e59-ca1b-4ae2-b520-6fd11121829e\") " pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106809 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83528e59-ca1b-4ae2-b520-6fd11121829e-konnectivity-ca\") pod \"konnectivity-agent-b8zxh\" (UID: \"83528e59-ca1b-4ae2-b520-6fd11121829e\") " pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106837 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-node-log\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106876 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106895 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfqcb\" (UniqueName: \"kubernetes.io/projected/ca0eef2d-7095-4da1-84cc-c0d803143be7-kube-api-access-sfqcb\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106910 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-cni-binary-copy\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106928 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-os-release\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106944 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-iptables-alerter-script\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.106961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106962 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wbmj\" (UniqueName: \"kubernetes.io/projected/fcb19980-477a-48cd-92bc-6d4554fab245-kube-api-access-5wbmj\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106977 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-kubelet\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.106993 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107009 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-etc-selinux\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107026 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-run-netns\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107042 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-cni-netd\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107058 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-os-release\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107074 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-multus-certs\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107089 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysctl-d\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107103 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03a0f57f-0f1b-4945-bc24-d2c547091546-tmp\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107118 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovnkube-script-lib\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107133 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-host-slash\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107158 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107174 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea5ae2be-b7bd-4758-a311-9104121792f1-dbus\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107191 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-kubelet\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107207 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-modprobe-d\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107224 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-slash\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107290 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107247 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-var-lib-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107263 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-env-overrides\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107280 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fdb909c-8893-43f5-ba58-a8f774076194-serviceca\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107297 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-k8s-cni-cncf-io\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107318 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-cni-multus\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107336 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-system-cni-dir\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107355 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cni-binary-copy\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107372 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-log-socket\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107388 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-cni-bin\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107407 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94jsq\" (UniqueName: \"kubernetes.io/projected/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-kube-api-access-94jsq\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107424 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovnkube-config\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107439 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysctl-conf\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107454 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-run\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107472 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-host\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107489 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-etc-kubernetes\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107516 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107533 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-sys\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.107774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107548 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-registration-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107563 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-sys-fs\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107565 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-kubelet\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107631 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-system-cni-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107637 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-run-netns\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107586 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-system-cni-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107672 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-cni-netd\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107673 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-node-log\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107678 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-cni-multus\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107708 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107712 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-log-socket\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107719 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-os-release\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107733 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-k8s-cni-cncf-io\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107746 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-cni-bin\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107773 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-etc-kubernetes\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107812 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-daemon-config\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107835 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107855 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-kubernetes\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.108347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107869 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-var-lib-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107896 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-ovn\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107925 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-var-lib-kubelet\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107930 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-ovn\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.107991 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-slash\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.108087 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.108153 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:34.60813282 +0000 UTC m=+2.933980088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108267 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-env-overrides\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108327 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-cni-bin\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108346 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovn-node-metrics-cert\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108376 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03f2645b-2fc8-4bef-9bec-128198246ad6-hosts-file\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108390 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-cni-bin\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108394 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmsnw\" (UniqueName: \"kubernetes.io/projected/03f2645b-2fc8-4bef-9bec-128198246ad6-kube-api-access-gmsnw\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108402 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fdb909c-8893-43f5-ba58-a8f774076194-serviceca\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108437 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-multus-certs\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108467 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-etc-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108491 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-cni-binary-copy\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.108992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108541 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-etc-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108543 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-socket-dir-parent\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108579 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-netns\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108597 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-hostroot\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108615 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-daemon-config\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108656 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-socket-dir-parent\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108714 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-host-run-netns\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108728 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-hostroot\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108746 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-conf-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108776 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-conf-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108775 2534 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108822 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-systemd\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108853 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cnibin\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108885 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-systemd-units\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108922 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovnkube-script-lib\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.108941 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109003 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-systemd-units\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109061 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-openvswitch\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.109732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109080 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109116 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fdb909c-8893-43f5-ba58-a8f774076194-host\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109150 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109203 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7mhj\" (UniqueName: \"kubernetes.io/projected/1fdb909c-8893-43f5-ba58-a8f774076194-kube-api-access-v7mhj\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109312 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-cnibin\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109342 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109422 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fdb909c-8893-43f5-ba58-a8f774076194-host\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109435 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-cnibin\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109469 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109495 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysconfig\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109526 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-var-lib-kubelet\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109552 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhkgm\" (UniqueName: \"kubernetes.io/projected/03a0f57f-0f1b-4945-bc24-d2c547091546-kube-api-access-jhkgm\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109580 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st55j\" (UniqueName: \"kubernetes.io/projected/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-kube-api-access-st55j\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109604 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109621 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-socket-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109636 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-device-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109676 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-systemd\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.110742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109692 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-cni-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.111595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109730 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njnpb\" (UniqueName: \"kubernetes.io/projected/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-kube-api-access-njnpb\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.111595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109826 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea5ae2be-b7bd-4758-a311-9104121792f1-kubelet-config\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.111595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109850 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-multus-cni-dir\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.111595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.109929 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca0eef2d-7095-4da1-84cc-c0d803143be7-run-systemd\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.113553 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.113531 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca0eef2d-7095-4da1-84cc-c0d803143be7-ovn-node-metrics-cert\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.126101 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.126069 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:49:34.126101 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.126099 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:49:34.126302 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.126128 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:34.126302 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.126215 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:34.626193212 +0000 UTC m=+2.952040478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:34.128733 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.128707 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfqcb\" (UniqueName: \"kubernetes.io/projected/ca0eef2d-7095-4da1-84cc-c0d803143be7-kube-api-access-sfqcb\") pod \"ovnkube-node-2g487\" (UID: \"ca0eef2d-7095-4da1-84cc-c0d803143be7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.129556 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.129528 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt5ct\" (UniqueName: \"kubernetes.io/projected/80dd74e9-d654-4fa1-98bf-2bcf2395da42-kube-api-access-mt5ct\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:34.130719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.130699 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njnpb\" (UniqueName: \"kubernetes.io/projected/f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a-kube-api-access-njnpb\") pod \"multus-8cdj6\" (UID: \"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a\") " pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.132003 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.131980 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7mhj\" (UniqueName: \"kubernetes.io/projected/1fdb909c-8893-43f5-ba58-a8f774076194-kube-api-access-v7mhj\") pod \"node-ca-4c6jg\" (UID: \"1fdb909c-8893-43f5-ba58-a8f774076194\") " pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.210535 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210502 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03f2645b-2fc8-4bef-9bec-128198246ad6-hosts-file\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210545 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmsnw\" (UniqueName: \"kubernetes.io/projected/03f2645b-2fc8-4bef-9bec-128198246ad6-kube-api-access-gmsnw\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210573 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-systemd\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210595 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cnibin\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210634 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210658 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysconfig\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210669 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-systemd\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210725 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03f2645b-2fc8-4bef-9bec-128198246ad6-hosts-file\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.210740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210742 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysconfig\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210679 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-var-lib-kubelet\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210747 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-var-lib-kubelet\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210786 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cnibin\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210838 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhkgm\" (UniqueName: \"kubernetes.io/projected/03a0f57f-0f1b-4945-bc24-d2c547091546-kube-api-access-jhkgm\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210885 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st55j\" (UniqueName: \"kubernetes.io/projected/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-kube-api-access-st55j\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210914 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210939 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-socket-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210965 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-device-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210972 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.210995 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea5ae2be-b7bd-4758-a311-9104121792f1-kubelet-config\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211012 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-tuned\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211019 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211029 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f2645b-2fc8-4bef-9bec-128198246ad6-tmp-dir\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211073 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-lib-modules\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211085 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-socket-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211098 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea5ae2be-b7bd-4758-a311-9104121792f1-kubelet-config\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.211181 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211100 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83528e59-ca1b-4ae2-b520-6fd11121829e-agent-certs\") pod \"konnectivity-agent-b8zxh\" (UID: \"83528e59-ca1b-4ae2-b520-6fd11121829e\") " pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211138 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-device-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211149 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83528e59-ca1b-4ae2-b520-6fd11121829e-konnectivity-ca\") pod \"konnectivity-agent-b8zxh\" (UID: \"83528e59-ca1b-4ae2-b520-6fd11121829e\") " pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211178 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-os-release\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211234 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-os-release\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211275 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-lib-modules\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211288 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f2645b-2fc8-4bef-9bec-128198246ad6-tmp-dir\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211304 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-iptables-alerter-script\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211331 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wbmj\" (UniqueName: \"kubernetes.io/projected/fcb19980-477a-48cd-92bc-6d4554fab245-kube-api-access-5wbmj\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211359 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211383 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-etc-selinux\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211412 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysctl-d\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211468 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03a0f57f-0f1b-4945-bc24-d2c547091546-tmp\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211523 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-host-slash\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.211537 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211549 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211576 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea5ae2be-b7bd-4758-a311-9104121792f1-dbus\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.212043 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.211601 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:34.711578817 +0000 UTC m=+3.037426069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211666 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-modprobe-d\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211704 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-system-cni-dir\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211748 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cni-binary-copy\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211775 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94jsq\" (UniqueName: \"kubernetes.io/projected/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-kube-api-access-94jsq\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211828 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysctl-conf\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211863 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83528e59-ca1b-4ae2-b520-6fd11121829e-konnectivity-ca\") pod \"konnectivity-agent-b8zxh\" (UID: \"83528e59-ca1b-4ae2-b520-6fd11121829e\") " pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211853 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-run\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211923 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-run\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211948 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-host\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211962 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-iptables-alerter-script\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211968 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-system-cni-dir\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.211991 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212018 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-modprobe-d\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212030 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-sys\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212060 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-registration-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212066 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysctl-d\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.212945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212071 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-host-slash\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212091 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-sys-fs\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212131 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-kubernetes\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212148 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-host\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212286 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-kubernetes\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212294 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-registration-dir\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212334 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-sys\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212358 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-etc-selinux\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212376 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-sysctl-conf\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212421 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcb19980-477a-48cd-92bc-6d4554fab245-sys-fs\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212521 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea5ae2be-b7bd-4758-a311-9104121792f1-dbus\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212569 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cni-binary-copy\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212665 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.214008 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.212751 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.215795 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.215729 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03a0f57f-0f1b-4945-bc24-d2c547091546-etc-tuned\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.215795 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.215753 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83528e59-ca1b-4ae2-b520-6fd11121829e-agent-certs\") pod \"konnectivity-agent-b8zxh\" (UID: \"83528e59-ca1b-4ae2-b520-6fd11121829e\") " pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.216912 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.216887 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03a0f57f-0f1b-4945-bc24-d2c547091546-tmp\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.226509 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.226427 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94jsq\" (UniqueName: \"kubernetes.io/projected/0f24d5f9-8caa-4d15-974f-7dd17f4cd111-kube-api-access-94jsq\") pod \"multus-additional-cni-plugins-q2zv9\" (UID: \"0f24d5f9-8caa-4d15-974f-7dd17f4cd111\") " pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.227285 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.227258 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wbmj\" (UniqueName: \"kubernetes.io/projected/fcb19980-477a-48cd-92bc-6d4554fab245-kube-api-access-5wbmj\") pod \"aws-ebs-csi-driver-node-nggkm\" (UID: \"fcb19980-477a-48cd-92bc-6d4554fab245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.227397 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.227255 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st55j\" (UniqueName: \"kubernetes.io/projected/d4f3f3c6-fca3-462b-88ea-8621f90d5af7-kube-api-access-st55j\") pod \"iptables-alerter-6cwcg\" (UID: \"d4f3f3c6-fca3-462b-88ea-8621f90d5af7\") " pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.228371 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.228344 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhkgm\" (UniqueName: \"kubernetes.io/projected/03a0f57f-0f1b-4945-bc24-d2c547091546-kube-api-access-jhkgm\") pod \"tuned-qmsn4\" (UID: \"03a0f57f-0f1b-4945-bc24-d2c547091546\") " pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.228811 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.228785 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmsnw\" (UniqueName: \"kubernetes.io/projected/03f2645b-2fc8-4bef-9bec-128198246ad6-kube-api-access-gmsnw\") pod \"node-resolver-ngkzd\" (UID: \"03f2645b-2fc8-4bef-9bec-128198246ad6\") " pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.305277 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.305227 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:49:34.309999 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.309977 2534 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:49:34.315623 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.315599 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4c6jg" Jul 22 11:49:34.327777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.327753 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8cdj6" Jul 22 11:49:34.335862 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.335837 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6cwcg" Jul 22 11:49:34.346857 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.346829 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:34.355329 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.355303 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" Jul 22 11:49:34.363376 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.363353 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngkzd" Jul 22 11:49:34.373445 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.373418 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" Jul 22 11:49:34.380559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.380535 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" Jul 22 11:49:34.614872 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.614773 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:34.615020 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.614944 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:34.615069 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.615020 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:35.614994673 +0000 UTC m=+3.940841917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:34.715655 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.715617 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:34.715882 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:34.715709 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:34.715941 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.715886 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:49:34.715941 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.715906 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:49:34.715941 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.715920 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:34.716059 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.715983 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:35.715962659 +0000 UTC m=+4.041809909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:34.716272 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.716237 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:34.716410 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:34.716315 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:35.716296682 +0000 UTC m=+4.042143929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:35.044993 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.044906 2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:44:34 +0000 UTC" deadline="2028-04-24 23:20:01.406516464 +0000 UTC" Jul 22 11:49:35.044993 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.044941 2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15419h30m26.361578857s" Jul 22 11:49:35.068331 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:35.068286 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10d6cdd_149b_4d6f_a97b_49cb55ce7b9a.slice/crio-74421a4566bdc380b2ede9640bd0b131547f09a484ff52132a302683e850a71b WatchSource:0}: Error finding container 74421a4566bdc380b2ede9640bd0b131547f09a484ff52132a302683e850a71b: Status 404 returned error can't find the container with id 74421a4566bdc380b2ede9640bd0b131547f09a484ff52132a302683e850a71b Jul 22 11:49:35.069388 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:35.069363 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f2645b_2fc8_4bef_9bec_128198246ad6.slice/crio-7ba61a86e7bfb02019fe8aaece47c10cc79023aaf2b50874d74c42f3ecaee0e4 WatchSource:0}: Error finding container 7ba61a86e7bfb02019fe8aaece47c10cc79023aaf2b50874d74c42f3ecaee0e4: Status 404 returned error can't find the container with id 7ba61a86e7bfb02019fe8aaece47c10cc79023aaf2b50874d74c42f3ecaee0e4 Jul 22 11:49:35.070815 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:35.070784 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83528e59_ca1b_4ae2_b520_6fd11121829e.slice/crio-b48563aa6e7c318be4a1e999be94bb5c97189b43a3900bf2f85401ef46ec805d WatchSource:0}: Error finding container b48563aa6e7c318be4a1e999be94bb5c97189b43a3900bf2f85401ef46ec805d: Status 404 returned error can't find the container with id b48563aa6e7c318be4a1e999be94bb5c97189b43a3900bf2f85401ef46ec805d Jul 22 11:49:35.074159 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:35.074129 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fdb909c_8893_43f5_ba58_a8f774076194.slice/crio-7d45dc7b6fe1f7b333c295c2e98c84e91f9f68b3d6dc095d94416562961fbc84 WatchSource:0}: Error finding container 7d45dc7b6fe1f7b333c295c2e98c84e91f9f68b3d6dc095d94416562961fbc84: Status 404 returned error can't find the container with id 7d45dc7b6fe1f7b333c295c2e98c84e91f9f68b3d6dc095d94416562961fbc84 Jul 22 11:49:35.105818 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.105777 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngkzd" event={"ID":"03f2645b-2fc8-4bef-9bec-128198246ad6","Type":"ContainerStarted","Data":"7ba61a86e7bfb02019fe8aaece47c10cc79023aaf2b50874d74c42f3ecaee0e4"} Jul 22 11:49:35.105985 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:35.105956 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0eef2d_7095_4da1_84cc_c0d803143be7.slice/crio-54c0beadaef3a413add5e790d5ee50cfcd7f0d31aa4ea3c6606674a7e118d3b8 WatchSource:0}: Error finding container 54c0beadaef3a413add5e790d5ee50cfcd7f0d31aa4ea3c6606674a7e118d3b8: Status 404 returned error can't find the container with id 54c0beadaef3a413add5e790d5ee50cfcd7f0d31aa4ea3c6606674a7e118d3b8 Jul 22 11:49:35.107064 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:49:35.107041 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f24d5f9_8caa_4d15_974f_7dd17f4cd111.slice/crio-8ce423871c9c87efb901e6fd143926d442ec88e7c9d804f4f1d6b70fd9431482 WatchSource:0}: Error finding container 8ce423871c9c87efb901e6fd143926d442ec88e7c9d804f4f1d6b70fd9431482: Status 404 returned error can't find the container with id 8ce423871c9c87efb901e6fd143926d442ec88e7c9d804f4f1d6b70fd9431482 Jul 22 11:49:35.107462 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.107371 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" event={"ID":"03a0f57f-0f1b-4945-bc24-d2c547091546","Type":"ContainerStarted","Data":"a50a9ad3d5f5eafde5cf87c668ef845aeae4c15bcdd3c9faa392cdbc5683bcab"} Jul 22 11:49:35.108574 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.108538 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4c6jg" event={"ID":"1fdb909c-8893-43f5-ba58-a8f774076194","Type":"ContainerStarted","Data":"7d45dc7b6fe1f7b333c295c2e98c84e91f9f68b3d6dc095d94416562961fbc84"} Jul 22 11:49:35.109920 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.109810 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b8zxh" event={"ID":"83528e59-ca1b-4ae2-b520-6fd11121829e","Type":"ContainerStarted","Data":"b48563aa6e7c318be4a1e999be94bb5c97189b43a3900bf2f85401ef46ec805d"} Jul 22 11:49:35.111316 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.111175 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8cdj6" event={"ID":"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a","Type":"ContainerStarted","Data":"74421a4566bdc380b2ede9640bd0b131547f09a484ff52132a302683e850a71b"} Jul 22 11:49:35.112384 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.112356 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" event={"ID":"fcb19980-477a-48cd-92bc-6d4554fab245","Type":"ContainerStarted","Data":"c68729daa40af9597afd22e23d0b8cb5f7d6569f64b33e72fdd767f04153cd04"} Jul 22 11:49:35.113500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.113451 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6cwcg" event={"ID":"d4f3f3c6-fca3-462b-88ea-8621f90d5af7","Type":"ContainerStarted","Data":"b201f3a6f901eee08bf2b708deb1f2233c7a3756ec050a71ffce49f982f9e945"} Jul 22 11:49:35.439436 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.439247 2534 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:49:35.624438 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.624276 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:35.624438 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.624415 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:35.624666 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.624491 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:37.62447202 +0000 UTC m=+5.950319266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:35.725233 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.725192 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:35.725419 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:35.725250 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:35.725419 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.725382 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:35.725535 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.725445 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:37.725425851 +0000 UTC m=+6.051273315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:35.726029 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.725906 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:49:35.726029 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.725935 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:49:35.726029 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.725948 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:35.726029 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:35.725995 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:37.725979301 +0000 UTC m=+6.051826549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:36.045770 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.045634 2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:44:34 +0000 UTC" deadline="2028-05-08 02:37:57.93456382 +0000 UTC" Jul 22 11:49:36.045770 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.045705 2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15734h48m21.888863367s" Jul 22 11:49:36.097381 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.096535 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:36.097381 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:36.096723 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:36.097381 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.097200 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:36.097381 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:36.097326 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:36.097751 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.097528 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:36.097751 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:36.097619 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:36.135925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.135817 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" event={"ID":"a1252a5c274066b1c97fa5a755cd7d35","Type":"ContainerStarted","Data":"3c6c92d061d5e575703c43a46b491c735adae5f32a182e6813e39ee22b6bcece"} Jul 22 11:49:36.146905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.146834 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerStarted","Data":"8ce423871c9c87efb901e6fd143926d442ec88e7c9d804f4f1d6b70fd9431482"} Jul 22 11:49:36.165305 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.164654 2534 generic.go:356] "Generic (PLEG): container finished" podID="1826b170d96cfa607cb3c14f722f823e" containerID="422b6c0a3ffd91fa4541576e3be9d6ee426521a08f95a97b300280d130a14f6c" exitCode=0 Jul 22 11:49:36.165305 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.164994 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" event={"ID":"1826b170d96cfa607cb3c14f722f823e","Type":"ContainerDied","Data":"422b6c0a3ffd91fa4541576e3be9d6ee426521a08f95a97b300280d130a14f6c"} Jul 22 11:49:36.178800 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.178434 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"54c0beadaef3a413add5e790d5ee50cfcd7f0d31aa4ea3c6606674a7e118d3b8"} Jul 22 11:49:36.184571 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:36.184518 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-104.ec2.internal" podStartSLOduration=2.184497183 podStartE2EDuration="2.184497183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:36.16448402 +0000 UTC m=+4.490331288" watchObservedRunningTime="2026-07-22 11:49:36.184497183 +0000 UTC m=+4.510344452" Jul 22 11:49:37.201449 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:37.201411 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" event={"ID":"1826b170d96cfa607cb3c14f722f823e","Type":"ContainerStarted","Data":"cee4d530472826a5defd23ad527bbf018d31bab42199aeee378e11917de36433"} Jul 22 11:49:37.643263 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:37.643173 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:37.643485 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.643376 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:37.643485 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.643446 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:41.643425402 +0000 UTC m=+9.969272645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:37.744592 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:37.744665 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.744805 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.744892 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:41.744871905 +0000 UTC m=+10.070719151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.745329 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.745349 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.745363 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:37.747944 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:37.745410 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:41.745393917 +0000 UTC m=+10.071241166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:38.101868 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:38.101772 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:38.102044 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:38.101930 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:38.102461 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:38.102439 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:38.102578 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:38.102551 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:38.102649 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:38.102618 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:38.102730 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:38.102708 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:40.098269 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:40.097397 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:40.098269 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:40.097533 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:40.098269 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:40.097983 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:40.098269 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:40.098101 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:40.098269 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:40.098151 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:40.098269 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:40.098217 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:41.679370 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:41.679337 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:41.679887 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.679475 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:41.679887 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.679556 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:49.67953393 +0000 UTC m=+18.005381196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:41.779818 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:41.779776 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:41.780034 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:41.779907 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:41.780034 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.780008 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:41.780171 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.780044 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:49:41.780171 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.780063 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:49:41.780171 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.780081 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:41.780171 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.780094 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:49.780069998 +0000 UTC m=+18.105917255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:41.780171 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:41.780136 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:49:49.780118042 +0000 UTC m=+18.105965307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:42.097015 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:42.096933 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:42.097184 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:42.097028 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:42.097470 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:42.097346 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:42.097470 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:42.097409 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:42.098033 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:42.097757 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:42.098033 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:42.097889 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:44.095126 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:44.095091 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:44.095621 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:44.095238 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:44.095621 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:44.095091 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:44.095621 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:44.095354 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:44.095621 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:44.095091 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:44.095621 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:44.095435 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:46.096061 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:46.096028 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:46.096559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:46.096028 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:46.096559 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:46.096168 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:46.096559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:46.096028 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:46.096559 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:46.096251 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:46.096559 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:46.096340 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:48.095986 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:48.095948 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:48.095986 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:48.095989 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:48.096486 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:48.095957 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:48.096486 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:48.096092 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:48.096486 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:48.096174 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:48.096486 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:48.096258 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:49.746726 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:49.746686 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:49.747222 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.746878 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:49.747222 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.746976 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:05.746955074 +0000 UTC m=+34.072802316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:49:49.847200 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:49.847157 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:49.847362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:49.847216 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:49.847362 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.847319 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:49.847362 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.847333 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:49:49.847362 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.847351 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:49:49.847362 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.847365 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:49.847563 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.847392 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:05.847375046 +0000 UTC m=+34.173222309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:49:49.847563 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:49.847416 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:05.847405912 +0000 UTC m=+34.173253155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:49:50.095846 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:50.095304 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:50.095846 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:50.095358 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:50.095846 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:50.095305 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:50.095846 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:50.095436 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:50.095846 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:50.095577 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:50.095846 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:50.095656 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:52.096274 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:52.096242 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:52.096833 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:52.096306 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:52.096984 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:52.096321 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:52.096984 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:52.096904 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:52.097106 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:52.097002 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:52.097106 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:52.097095 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:54.095328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:54.095285 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:54.095780 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:54.095285 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:54.095780 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:54.095428 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:54.095780 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:54.095564 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:54.095780 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:54.095291 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:54.095780 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:54.095662 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:56.096150 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.095736 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:56.097022 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:56.096173 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:56.097022 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.095780 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:56.097022 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:56.096236 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:56.097022 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.095736 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:56.097022 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:56.096290 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:56.241813 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.241760 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" event={"ID":"fcb19980-477a-48cd-92bc-6d4554fab245","Type":"ContainerStarted","Data":"3c6a47e7e20c788eb000b764715b7c0cd2e1725d400050d6e4f298204ab9bb3d"} Jul 22 11:49:56.242997 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.242976 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngkzd" event={"ID":"03f2645b-2fc8-4bef-9bec-128198246ad6","Type":"ContainerStarted","Data":"7716deec7692b928186224fabfa1bc2ecd0cddeda50804fe6859796ec60b3c8c"} Jul 22 11:49:56.245631 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.245602 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"7b2011d918aa1bed0ba7424f973f875e978712998089fc0e078c511d3abea661"} Jul 22 11:49:56.245742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.245659 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"031076d78c48b4c65773d51e009c359c288208d5272968a2ecbf3d72c9f1a775"} Jul 22 11:49:56.245742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.245688 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"55595010fc083bdb056a81f9041d81b31cba2287c3357489c5ed5facb2236e50"} Jul 22 11:49:56.245742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.245718 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"c67e09a9621917b193c37c2b348ad348b3779abd1b9ea3169faca5c6931d488a"} Jul 22 11:49:56.245742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.245744 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"41d49311bab3da2e186d17dbca98d9d4a5bd30eff55e4870a8a4d72fbf4e7db5"} Jul 22 11:49:56.245940 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.245770 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"49d3043fd3e643bf328faa12b323ecf8c0218cc798ef166109aca2c5cf5c7eb0"} Jul 22 11:49:56.246989 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.246967 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" event={"ID":"03a0f57f-0f1b-4945-bc24-d2c547091546","Type":"ContainerStarted","Data":"fb80714effad03bc36247a03bae274b5a962b5166e9ea26c4904efbbc1c9be47"} Jul 22 11:49:56.248274 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.248245 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4c6jg" event={"ID":"1fdb909c-8893-43f5-ba58-a8f774076194","Type":"ContainerStarted","Data":"817334ac18d7fdb17a8a271d26b65775e4754067b3dbea6ba1fc81b2ca5d93a6"} Jul 22 11:49:56.249512 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.249491 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b8zxh" event={"ID":"83528e59-ca1b-4ae2-b520-6fd11121829e","Type":"ContainerStarted","Data":"33456ef226993a2f1c4a1bd101be753db21beb78d963c497445b3bf5b561068d"} Jul 22 11:49:56.250757 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.250734 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8cdj6" event={"ID":"f10d6cdd-149b-4d6f-a97b-49cb55ce7b9a","Type":"ContainerStarted","Data":"20364947521da014c0d0fb395451214b85660e7bb76a43d150c95e7c7f90dfba"} Jul 22 11:49:56.251985 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.251966 2534 generic.go:356] "Generic (PLEG): container finished" podID="0f24d5f9-8caa-4d15-974f-7dd17f4cd111" containerID="3eb7ea7fd3bcc389132a4cc4d87d8640ec41721cde9b6bd26e44fede0775c815" exitCode=0 Jul 22 11:49:56.252055 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.251993 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerDied","Data":"3eb7ea7fd3bcc389132a4cc4d87d8640ec41721cde9b6bd26e44fede0775c815"} Jul 22 11:49:56.262559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.262511 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-104.ec2.internal" podStartSLOduration=22.262480661 podStartE2EDuration="22.262480661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:37.222320208 +0000 UTC m=+5.548167477" watchObservedRunningTime="2026-07-22 11:49:56.262480661 +0000 UTC m=+24.588327903" Jul 22 11:49:56.262710 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.262691 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ngkzd" podStartSLOduration=3.036116247 podStartE2EDuration="23.262685054s" totalImagesPullingTime="20.226568807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:56.262457882 +0000 UTC m=+24.588305145" watchObservedRunningTime="2026-07-22 11:49:56.262685054 +0000 UTC m=+24.588532318" Jul 22 11:49:56.301724 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.301655 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4c6jg" podStartSLOduration=3.088057817 podStartE2EDuration="23.301639773s" totalImagesPullingTime="20.213581956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:56.277897642 +0000 UTC m=+24.603744917" watchObservedRunningTime="2026-07-22 11:49:56.301639773 +0000 UTC m=+24.627487038" Jul 22 11:49:56.302044 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.302002 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8cdj6" podStartSLOduration=3.023271772 podStartE2EDuration="23.301993235s" totalImagesPullingTime="20.278721463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:56.30147363 +0000 UTC m=+24.627320898" watchObservedRunningTime="2026-07-22 11:49:56.301993235 +0000 UTC m=+24.627840499" Jul 22 11:49:56.368079 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.367975 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qmsn4" podStartSLOduration=3.166072841 podStartE2EDuration="23.367952402s" totalImagesPullingTime="20.201879561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:56.350489003 +0000 UTC m=+24.676336268" watchObservedRunningTime="2026-07-22 11:49:56.367952402 +0000 UTC m=+24.693799666" Jul 22 11:49:56.368608 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.368575 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b8zxh" podStartSLOduration=7.367080896 podStartE2EDuration="23.368564308s" totalImagesPullingTime="16.001483412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:56.367894971 +0000 UTC m=+24.693742234" watchObservedRunningTime="2026-07-22 11:49:56.368564308 +0000 UTC m=+24.694411572" Jul 22 11:49:56.962679 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:56.962647 2534 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 11:49:57.083172 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:57.083022 2534 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T11:49:56.962675068Z","UUID":"1cc8d3d8-f216-4edb-8030-45b6029c4e7c","Handler":null,"Name":"","Endpoint":""} Jul 22 11:49:57.084994 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:57.084975 2534 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 11:49:57.085120 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:57.085013 2534 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 11:49:57.256352 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:57.256299 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" event={"ID":"fcb19980-477a-48cd-92bc-6d4554fab245","Type":"ContainerStarted","Data":"83bf29d7edad3262d91eae7a3e72428d8db909b20e8988ab10d10d15104df647"} Jul 22 11:49:57.258499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:57.258465 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6cwcg" event={"ID":"d4f3f3c6-fca3-462b-88ea-8621f90d5af7","Type":"ContainerStarted","Data":"b2886215aa9d646196e63e6216d4a04a99b58cb07d4575d1e2b6e6250f8a4363"} Jul 22 11:49:57.277001 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:57.276948 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6cwcg" podStartSLOduration=4.064404502 podStartE2EDuration="24.276928507s" totalImagesPullingTime="20.212524005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:57.276259853 +0000 UTC m=+25.602107117" watchObservedRunningTime="2026-07-22 11:49:57.276928507 +0000 UTC m=+25.602775774" Jul 22 11:49:58.095278 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:58.095245 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:49:58.095484 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:58.095298 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:49:58.095484 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:58.095392 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:49:58.095484 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:58.095435 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:49:58.095484 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:58.095484 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:49:58.095692 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:49:58.095556 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:49:58.263499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:58.263456 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" event={"ID":"fcb19980-477a-48cd-92bc-6d4554fab245","Type":"ContainerStarted","Data":"371efcc1298fb04f5135f8007bfcd7b97cbff979e79f6415fedaeee2a9facecc"} Jul 22 11:49:58.267514 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:58.267470 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"7f553c742ffefc44003472641d2d2039a4486551de29ce9beb54c3be98c4aea6"} Jul 22 11:49:58.282922 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:58.282869 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nggkm" podStartSLOduration=2.604560367 podStartE2EDuration="25.282852841s" totalImagesPullingTime="22.678292474s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:49:58.282748739 +0000 UTC m=+26.608596011" watchObservedRunningTime="2026-07-22 11:49:58.282852841 +0000 UTC m=+26.608700106" Jul 22 11:49:59.618871 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:59.618669 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:49:59.619403 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:49:59.619322 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:50:00.095130 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:00.095060 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:00.095130 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:00.095086 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:00.095352 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:00.095190 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:00.095352 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:00.095202 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:50:00.095352 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:00.095308 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:50:00.095465 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:00.095423 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:50:00.271856 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:00.271824 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:50:00.272588 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:00.272527 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:50:01.277285 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:01.276993 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" event={"ID":"ca0eef2d-7095-4da1-84cc-c0d803143be7","Type":"ContainerStarted","Data":"331e1b92d7e47909ea4a415f655a2fcd1a5c2028eab445af41f339eaed33a26f"} Jul 22 11:50:01.277285 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:01.277240 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:50:01.278885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:01.278863 2534 generic.go:356] "Generic (PLEG): container finished" podID="0f24d5f9-8caa-4d15-974f-7dd17f4cd111" containerID="6d57cbf1ce002a6a88f4044dd8f9fe3f4e88c505ec29373e39c1276c630c38e9" exitCode=0 Jul 22 11:50:01.278998 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:01.278945 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerDied","Data":"6d57cbf1ce002a6a88f4044dd8f9fe3f4e88c505ec29373e39c1276c630c38e9"} Jul 22 11:50:01.293324 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:01.293304 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:50:01.343974 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:01.343916 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" podStartSLOduration=8.092940492 podStartE2EDuration="28.343899896s" totalImagesPullingTime="20.250959404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:01.310179932 +0000 UTC m=+29.636027220" watchObservedRunningTime="2026-07-22 11:50:01.343899896 +0000 UTC m=+29.669747181" Jul 22 11:50:02.095900 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.095868 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:02.096191 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:02.095996 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:50:02.096191 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.096035 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:02.096191 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.096081 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:02.096368 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:02.096189 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:50:02.096368 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:02.096305 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:50:02.282056 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.282028 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:50:02.282487 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.282093 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:50:02.298012 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.297985 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:50:02.648045 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.648015 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sw68n"] Jul 22 11:50:02.648225 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.648147 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:02.648287 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:02.648267 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:50:02.648756 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.648734 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gbj2"] Jul 22 11:50:02.648865 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.648825 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:02.648914 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:02.648900 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:50:02.651825 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.651785 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkgxt"] Jul 22 11:50:02.651932 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:02.651874 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:02.651985 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:02.651939 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:50:03.286278 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:03.286246 2534 generic.go:356] "Generic (PLEG): container finished" podID="0f24d5f9-8caa-4d15-974f-7dd17f4cd111" containerID="8838aab0605f117efacdf035f4b584822ccd22ea44fae84268a1bb53a5aeb3fa" exitCode=0 Jul 22 11:50:03.286740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:03.286323 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerDied","Data":"8838aab0605f117efacdf035f4b584822ccd22ea44fae84268a1bb53a5aeb3fa"} Jul 22 11:50:04.095525 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:04.095495 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:04.095525 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:04.095532 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:04.095852 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:04.095621 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:50:04.095852 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:04.095743 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:50:05.095491 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:05.095416 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:05.095872 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.095505 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:50:05.292782 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:05.292747 2534 generic.go:356] "Generic (PLEG): container finished" podID="0f24d5f9-8caa-4d15-974f-7dd17f4cd111" containerID="cbaf6513d2f53462ae9c91b636e4efbc73710d23c431d916a84869b52467e6ea" exitCode=0 Jul 22 11:50:05.292969 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:05.292836 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerDied","Data":"cbaf6513d2f53462ae9c91b636e4efbc73710d23c431d916a84869b52467e6ea"} Jul 22 11:50:05.770468 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:05.770430 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:05.770647 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.770607 2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:50:05.770701 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.770688 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs podName:80dd74e9-d654-4fa1-98bf-2bcf2395da42 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:37.770666842 +0000 UTC m=+66.096514091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs") pod "network-metrics-daemon-sw68n" (UID: "80dd74e9-d654-4fa1-98bf-2bcf2395da42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:50:05.871328 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:05.871296 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:05.871505 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:05.871348 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:05.871505 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.871471 2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:50:05.871505 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.871490 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:50:05.871669 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.871510 2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:50:05.871669 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.871524 2534 projected.go:196] Error preparing data for projected volume kube-api-access-d62f4 for pod openshift-network-diagnostics/network-check-target-9gbj2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:50:05.871669 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.871536 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret podName:ea5ae2be-b7bd-4758-a311-9104121792f1 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:37.871518203 +0000 UTC m=+66.197365449 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret") pod "global-pull-secret-syncer-dkgxt" (UID: "ea5ae2be-b7bd-4758-a311-9104121792f1") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:50:05.871669 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:05.871579 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4 podName:94604095-766a-474c-9742-261c38fe2645 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:37.871564075 +0000 UTC m=+66.197411320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d62f4" (UniqueName: "kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4") pod "network-check-target-9gbj2" (UID: "94604095-766a-474c-9742-261c38fe2645") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:50:06.095586 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:06.095555 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:06.095996 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:06.095598 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:06.095996 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:06.095716 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sw68n" podUID="80dd74e9-d654-4fa1-98bf-2bcf2395da42" Jul 22 11:50:06.095996 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:06.095858 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gbj2" podUID="94604095-766a-474c-9742-261c38fe2645" Jul 22 11:50:07.095284 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:07.095204 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:07.095429 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:07.095328 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkgxt" podUID="ea5ae2be-b7bd-4758-a311-9104121792f1" Jul 22 11:50:08.043469 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.043439 2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-104.ec2.internal" event="NodeReady" Jul 22 11:50:08.044115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.043592 2534 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 11:50:08.083865 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.083770 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-578cbd896f-dfpt2"] Jul 22 11:50:08.089411 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.089388 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.092188 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.092170 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 11:50:08.092298 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.092218 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f7k7c\"" Jul 22 11:50:08.092362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.092305 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 11:50:08.092412 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.092175 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 11:50:08.095751 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.094917 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:08.095751 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.095161 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:08.098069 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.098045 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mjncz\"" Jul 22 11:50:08.098301 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.098264 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 11:50:08.098923 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.098601 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 11:50:08.098923 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.098603 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 11:50:08.099326 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.099167 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b6dpz\"" Jul 22 11:50:08.099797 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.099775 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 11:50:08.100186 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.100167 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-578cbd896f-dfpt2"] Jul 22 11:50:08.102132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.102112 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k494z"] Jul 22 11:50:08.106542 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.106521 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wm925"] Jul 22 11:50:08.107190 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.107173 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.109716 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.109698 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 11:50:08.109834 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.109700 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 11:50:08.109997 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.109974 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2xm6s\"" Jul 22 11:50:08.110486 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.110468 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.113269 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.113126 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 11:50:08.113379 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.113364 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wgjkp\"" Jul 22 11:50:08.113748 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.113618 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 11:50:08.114002 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.113982 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 11:50:08.117755 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.117737 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k494z"] Jul 22 11:50:08.118533 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.118502 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wm925"] Jul 22 11:50:08.189423 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189392 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-bound-sa-token\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189423 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189425 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.189630 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189495 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05e907e6-7648-4881-9d40-a4ffffd9d2c5-cert\") pod \"ingress-canary-wm925\" (UID: \"05e907e6-7648-4881-9d40-a4ffffd9d2c5\") " pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.189630 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189532 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-image-registry-private-configuration\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189630 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189603 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-installation-pull-secrets\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189776 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189636 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-certificates\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189776 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189653 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsgl2\" (UniqueName: \"kubernetes.io/projected/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-kube-api-access-lsgl2\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.189776 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189705 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jffr4\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-kube-api-access-jffr4\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189776 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189743 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-config-volume\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.189974 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189775 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5kgv\" (UniqueName: \"kubernetes.io/projected/05e907e6-7648-4881-9d40-a4ffffd9d2c5-kube-api-access-v5kgv\") pod \"ingress-canary-wm925\" (UID: \"05e907e6-7648-4881-9d40-a4ffffd9d2c5\") " pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.189974 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189818 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-ca-trust-extracted\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189974 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189866 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189974 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189910 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-trusted-ca\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.189974 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.189936 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-tmp-dir\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.290852 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.290782 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05e907e6-7648-4881-9d40-a4ffffd9d2c5-cert\") pod \"ingress-canary-wm925\" (UID: \"05e907e6-7648-4881-9d40-a4ffffd9d2c5\") " pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.291044 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.290862 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-image-registry-private-configuration\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291044 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.290895 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-installation-pull-secrets\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291044 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.290925 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-certificates\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291044 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.290951 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsgl2\" (UniqueName: \"kubernetes.io/projected/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-kube-api-access-lsgl2\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291653 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jffr4\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-kube-api-access-jffr4\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291694 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-config-volume\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291721 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5kgv\" (UniqueName: \"kubernetes.io/projected/05e907e6-7648-4881-9d40-a4ffffd9d2c5-kube-api-access-v5kgv\") pod \"ingress-canary-wm925\" (UID: \"05e907e6-7648-4881-9d40-a4ffffd9d2c5\") " pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291746 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-ca-trust-extracted\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291780 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291832 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-trusted-ca\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291853 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-tmp-dir\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291915 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-bound-sa-token\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.291929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.291936 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.292347 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.292045 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:50:08.292347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.292073 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-certificates\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.292347 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.292104 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls podName:50f09e4c-b2c9-40e7-a509-278f3b5fcb63 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:08.792087019 +0000 UTC m=+37.117934279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls") pod "dns-default-k494z" (UID: "50f09e4c-b2c9-40e7-a509-278f3b5fcb63") : secret "dns-default-metrics-tls" not found Jul 22 11:50:08.292347 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.292165 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 11:50:08.292347 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.292178 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-578cbd896f-dfpt2: secret "image-registry-tls" not found Jul 22 11:50:08.292347 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.292230 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls podName:6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a nodeName:}" failed. No retries permitted until 2026-07-22 11:50:08.79221563 +0000 UTC m=+37.118062891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls") pod "image-registry-578cbd896f-dfpt2" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a") : secret "image-registry-tls" not found Jul 22 11:50:08.293534 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.293058 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-config-volume\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.293534 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.293439 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-trusted-ca\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.293534 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.293479 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-ca-trust-extracted\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.293745 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.293693 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-tmp-dir\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.295995 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.295973 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-image-registry-private-configuration\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.296085 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.296030 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05e907e6-7648-4881-9d40-a4ffffd9d2c5-cert\") pod \"ingress-canary-wm925\" (UID: \"05e907e6-7648-4881-9d40-a4ffffd9d2c5\") " pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.297212 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.297193 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-installation-pull-secrets\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.301702 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.301680 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsgl2\" (UniqueName: \"kubernetes.io/projected/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-kube-api-access-lsgl2\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.302430 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.302405 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jffr4\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-kube-api-access-jffr4\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.302550 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.302466 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-bound-sa-token\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.302920 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.302896 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5kgv\" (UniqueName: \"kubernetes.io/projected/05e907e6-7648-4881-9d40-a4ffffd9d2c5-kube-api-access-v5kgv\") pod \"ingress-canary-wm925\" (UID: \"05e907e6-7648-4881-9d40-a4ffffd9d2c5\") " pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.435613 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.435524 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wm925" Jul 22 11:50:08.795649 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.795554 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:08.795834 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.795659 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:08.795834 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.795688 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:50:08.795834 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.795781 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls podName:50f09e4c-b2c9-40e7-a509-278f3b5fcb63 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:09.795761156 +0000 UTC m=+38.121608412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls") pod "dns-default-k494z" (UID: "50f09e4c-b2c9-40e7-a509-278f3b5fcb63") : secret "dns-default-metrics-tls" not found Jul 22 11:50:08.795834 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.795810 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 11:50:08.795834 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.795825 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-578cbd896f-dfpt2: secret "image-registry-tls" not found Jul 22 11:50:08.796143 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:08.795880 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls podName:6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a nodeName:}" failed. No retries permitted until 2026-07-22 11:50:09.795862717 +0000 UTC m=+38.121709964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls") pod "image-registry-578cbd896f-dfpt2" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a") : secret "image-registry-tls" not found Jul 22 11:50:08.894893 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.894860 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n"] Jul 22 11:50:08.898756 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.898733 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79ccd9f656-4kgpj"] Jul 22 11:50:08.898923 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.898879 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:08.901454 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.901431 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 11:50:08.901651 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.901637 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 11:50:08.901940 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.901922 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wvtm7\"" Jul 22 11:50:08.902028 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.902007 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 11:50:08.902495 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.902476 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-656k7"] Jul 22 11:50:08.902989 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.902971 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:08.903377 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.903202 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 11:50:08.905410 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.905394 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vljkf\"" Jul 22 11:50:08.906037 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.906015 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.906483 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.906222 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 11:50:08.906483 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.906229 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 11:50:08.907020 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.907002 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 11:50:08.907112 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.907030 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 11:50:08.908318 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.908259 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 11:50:08.908549 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.908533 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 11:50:08.909552 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.909531 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n"] Jul 22 11:50:08.911405 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.910322 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 11:50:08.911405 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.910986 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 11:50:08.911405 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.911187 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pbx4p\"" Jul 22 11:50:08.911405 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.911274 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 11:50:08.911625 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.911455 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 11:50:08.911625 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.911612 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79ccd9f656-4kgpj"] Jul 22 11:50:08.913201 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.912558 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-656k7"] Jul 22 11:50:08.918284 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.918258 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 11:50:08.960837 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.960517 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw"] Jul 22 11:50:08.964251 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.964222 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" Jul 22 11:50:08.968735 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.968702 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:50:08.968735 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.968728 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 11:50:08.968977 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.968761 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sgft2\"" Jul 22 11:50:08.973928 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.973903 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw"] Jul 22 11:50:08.992260 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.992225 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m"] Jul 22 11:50:08.996002 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.995971 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-74m8f"] Jul 22 11:50:08.996155 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996059 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:08.996749 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996726 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-stats-auth\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:08.996874 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996757 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-snapshots\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.996874 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996779 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7tvn\" (UniqueName: \"kubernetes.io/projected/8b58a72c-7ded-4849-a1fb-df32c70b6081-kube-api-access-n7tvn\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:08.996874 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996820 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-default-certificate\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:08.997031 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996884 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:08.997031 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.996997 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.997116 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997027 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:08.997116 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997058 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcp95\" (UniqueName: \"kubernetes.io/projected/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-kube-api-access-rcp95\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.997116 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997097 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttjjt\" (UniqueName: \"kubernetes.io/projected/ba13c544-b6f5-4b80-92c8-3f672a89de5e-kube-api-access-ttjjt\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:08.997208 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997116 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:08.997208 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997138 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-service-ca-bundle\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.997208 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997185 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ba13c544-b6f5-4b80-92c8-3f672a89de5e-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:08.997344 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997217 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-tmp\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.997344 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.997296 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-serving-cert\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:08.999407 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.999383 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44"] Jul 22 11:50:08.999532 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:08.999512 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.002756 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.002736 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx"] Jul 22 11:50:09.002911 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.002893 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.004183 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.004064 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 11:50:09.004958 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.004759 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jnbgt\"" Jul 22 11:50:09.005086 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.005071 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 11:50:09.005682 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.005666 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:50:09.006186 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.006171 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.006685 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.006662 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 11:50:09.010330 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.010313 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ttq67\"" Jul 22 11:50:09.014045 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.013850 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 11:50:09.014045 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.013873 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 11:50:09.014294 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.014276 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 11:50:09.015710 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.015689 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 11:50:09.017047 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.017028 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:50:09.021004 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.020986 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:50:09.021675 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.021241 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 11:50:09.021675 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.021387 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 11:50:09.021675 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.021471 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-74dnh\"" Jul 22 11:50:09.021675 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.021552 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pfjph\"" Jul 22 11:50:09.021912 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.021779 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 11:50:09.021912 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.021814 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:50:09.023665 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.023642 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44"] Jul 22 11:50:09.025413 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.025314 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx"] Jul 22 11:50:09.027961 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.027938 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-74m8f"] Jul 22 11:50:09.028063 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.027988 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m"] Jul 22 11:50:09.028693 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.028676 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 11:50:09.043439 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.043413 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 11:50:09.095194 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.095124 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:09.097759 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097731 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 11:50:09.097759 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097740 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-tmp\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.097954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097772 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-serving-cert\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.097954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097809 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5071ae6-69e3-42fb-9b62-e35c5e89017f-config\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.097954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097833 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2be84eee-fb05-4086-9c78-d70bf3a3f00e-serving-cert\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.097954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097857 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:09.097954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097892 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-stats-auth\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.097954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097916 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-snapshots\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097957 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2smrp\" (UniqueName: \"kubernetes.io/projected/dea87c46-b392-4a6d-97cb-116c1d213d84-kube-api-access-2smrp\") pod \"volume-data-source-validator-78cf4c6f6-x6bsw\" (UID: \"dea87c46-b392-4a6d-97cb-116c1d213d84\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.097981 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098018 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7tvn\" (UniqueName: \"kubernetes.io/projected/8b58a72c-7ded-4849-a1fb-df32c70b6081-kube-api-access-n7tvn\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098042 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qrt8\" (UniqueName: \"kubernetes.io/projected/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-kube-api-access-4qrt8\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098060 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2be84eee-fb05-4086-9c78-d70bf3a3f00e-config\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098086 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-default-certificate\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098111 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098134 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2be84eee-fb05-4086-9c78-d70bf3a3f00e-trusted-ca\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.098192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098173 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zz2p\" (UniqueName: \"kubernetes.io/projected/a5071ae6-69e3-42fb-9b62-e35c5e89017f-kube-api-access-5zz2p\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098198 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2b4d\" (UniqueName: \"kubernetes.io/projected/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-kube-api-access-m2b4d\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098232 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098258 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098280 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkkx8\" (UniqueName: \"kubernetes.io/projected/2be84eee-fb05-4086-9c78-d70bf3a3f00e-kube-api-access-rkkx8\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098306 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098333 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcp95\" (UniqueName: \"kubernetes.io/projected/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-kube-api-access-rcp95\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098378 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttjjt\" (UniqueName: \"kubernetes.io/projected/ba13c544-b6f5-4b80-92c8-3f672a89de5e-kube-api-access-ttjjt\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098403 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098427 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-service-ca-bundle\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098450 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5071ae6-69e3-42fb-9b62-e35c5e89017f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098508 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ba13c544-b6f5-4b80-92c8-3f672a89de5e-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:09.098521 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098518 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-snapshots\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.098964 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.098844 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-tmp\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.099244 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.099225 2534 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:50:09.099315 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.099292 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:09.59927392 +0000 UTC m=+37.925121165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : secret "router-metrics-certs-default" not found Jul 22 11:50:09.099525 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.099508 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:09.599491874 +0000 UTC m=+37.925339139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : configmap references non-existent config key: service-ca.crt Jul 22 11:50:09.099604 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.099590 2534 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:09.099667 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.099633 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls podName:ba13c544-b6f5-4b80-92c8-3f672a89de5e nodeName:}" failed. No retries permitted until 2026-07-22 11:50:09.599620889 +0000 UTC m=+37.925468134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-brh7n" (UID: "ba13c544-b6f5-4b80-92c8-3f672a89de5e") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:09.099827 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.099753 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.100236 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.100217 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-service-ca-bundle\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.100332 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.100316 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ba13c544-b6f5-4b80-92c8-3f672a89de5e-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:09.101736 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.101717 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-serving-cert\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.101856 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.101820 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-stats-auth\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.102562 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.102542 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-default-certificate\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.111903 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.111872 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcp95\" (UniqueName: \"kubernetes.io/projected/ecf76870-fbfb-4e3f-80ad-7c83e7540f88-kube-api-access-rcp95\") pod \"insights-operator-555d9d7885-656k7\" (UID: \"ecf76870-fbfb-4e3f-80ad-7c83e7540f88\") " pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.112904 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.112874 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttjjt\" (UniqueName: \"kubernetes.io/projected/ba13c544-b6f5-4b80-92c8-3f672a89de5e-kube-api-access-ttjjt\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:09.114410 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.114385 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7tvn\" (UniqueName: \"kubernetes.io/projected/8b58a72c-7ded-4849-a1fb-df32c70b6081-kube-api-access-n7tvn\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.199407 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199371 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2smrp\" (UniqueName: \"kubernetes.io/projected/dea87c46-b392-4a6d-97cb-116c1d213d84-kube-api-access-2smrp\") pod \"volume-data-source-validator-78cf4c6f6-x6bsw\" (UID: \"dea87c46-b392-4a6d-97cb-116c1d213d84\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" Jul 22 11:50:09.199407 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199408 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199428 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qrt8\" (UniqueName: \"kubernetes.io/projected/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-kube-api-access-4qrt8\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199447 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2be84eee-fb05-4086-9c78-d70bf3a3f00e-config\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199473 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2be84eee-fb05-4086-9c78-d70bf3a3f00e-trusted-ca\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199502 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zz2p\" (UniqueName: \"kubernetes.io/projected/a5071ae6-69e3-42fb-9b62-e35c5e89017f-kube-api-access-5zz2p\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199524 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2b4d\" (UniqueName: \"kubernetes.io/projected/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-kube-api-access-m2b4d\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199563 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199580 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkkx8\" (UniqueName: \"kubernetes.io/projected/2be84eee-fb05-4086-9c78-d70bf3a3f00e-kube-api-access-rkkx8\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.199642 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199643 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5071ae6-69e3-42fb-9b62-e35c5e89017f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.199999 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199686 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5071ae6-69e3-42fb-9b62-e35c5e89017f-config\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.199999 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199709 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2be84eee-fb05-4086-9c78-d70bf3a3f00e-serving-cert\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.199999 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.199726 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:09.201194 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.201145 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 11:50:09.201322 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.201247 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls podName:a3b8eaf6-c269-473e-ad55-a76a50ad06e9 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:09.701216939 +0000 UTC m=+38.027064182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-r8f4m" (UID: "a3b8eaf6-c269-473e-ad55-a76a50ad06e9") : secret "samples-operator-tls" not found Jul 22 11:50:09.201559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.201530 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2be84eee-fb05-4086-9c78-d70bf3a3f00e-trusted-ca\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.201559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.201538 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2be84eee-fb05-4086-9c78-d70bf3a3f00e-config\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.201920 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.201889 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5071ae6-69e3-42fb-9b62-e35c5e89017f-config\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.206201 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.202311 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.206201 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.202766 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.206201 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.202888 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5071ae6-69e3-42fb-9b62-e35c5e89017f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.206201 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.204260 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2be84eee-fb05-4086-9c78-d70bf3a3f00e-serving-cert\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.209885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.209858 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qrt8\" (UniqueName: \"kubernetes.io/projected/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-kube-api-access-4qrt8\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:09.209987 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.209972 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2smrp\" (UniqueName: \"kubernetes.io/projected/dea87c46-b392-4a6d-97cb-116c1d213d84-kube-api-access-2smrp\") pod \"volume-data-source-validator-78cf4c6f6-x6bsw\" (UID: \"dea87c46-b392-4a6d-97cb-116c1d213d84\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" Jul 22 11:50:09.210578 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.210540 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zz2p\" (UniqueName: \"kubernetes.io/projected/a5071ae6-69e3-42fb-9b62-e35c5e89017f-kube-api-access-5zz2p\") pod \"service-ca-operator-5bcbc94c78-nvc44\" (UID: \"a5071ae6-69e3-42fb-9b62-e35c5e89017f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.210926 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.210905 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2b4d\" (UniqueName: \"kubernetes.io/projected/0ba8d0cb-2b77-48d8-b186-b4152f82d6f8-kube-api-access-m2b4d\") pod \"kube-storage-version-migrator-operator-7658d9b749-nwgcx\" (UID: \"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.211334 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.211312 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkkx8\" (UniqueName: \"kubernetes.io/projected/2be84eee-fb05-4086-9c78-d70bf3a3f00e-kube-api-access-rkkx8\") pod \"console-operator-565b7b569d-74m8f\" (UID: \"2be84eee-fb05-4086-9c78-d70bf3a3f00e\") " pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.230451 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.230421 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-656k7" Jul 22 11:50:09.278705 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.278478 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" Jul 22 11:50:09.314923 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.314897 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:09.329449 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.329059 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" Jul 22 11:50:09.336046 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.335817 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" Jul 22 11:50:09.387944 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.386826 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2"] Jul 22 11:50:09.403867 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.403471 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2"] Jul 22 11:50:09.403867 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.403599 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" Jul 22 11:50:09.411931 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.411256 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pght7\"" Jul 22 11:50:09.484103 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.484045 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-656k7"] Jul 22 11:50:09.488110 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.487602 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wm925"] Jul 22 11:50:09.496991 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:09.496456 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e907e6_7648_4881_9d40_a4ffffd9d2c5.slice/crio-4ba3f081d30825ff8b2ad688105fed5daab971abd5ca94ddcad360fa01e19888 WatchSource:0}: Error finding container 4ba3f081d30825ff8b2ad688105fed5daab971abd5ca94ddcad360fa01e19888: Status 404 returned error can't find the container with id 4ba3f081d30825ff8b2ad688105fed5daab971abd5ca94ddcad360fa01e19888 Jul 22 11:50:09.512251 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.512135 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf29f\" (UniqueName: \"kubernetes.io/projected/3d00f959-4884-48cb-a81f-db9a8b81645c-kube-api-access-lf29f\") pod \"network-check-source-7bc5b6bf85-7vrx2\" (UID: \"3d00f959-4884-48cb-a81f-db9a8b81645c\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" Jul 22 11:50:09.549725 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.549694 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw"] Jul 22 11:50:09.567656 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.567630 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-74m8f"] Jul 22 11:50:09.572875 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.572849 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx"] Jul 22 11:50:09.587069 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.587039 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44"] Jul 22 11:50:09.591993 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:09.591962 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba8d0cb_2b77_48d8_b186_b4152f82d6f8.slice/crio-e6f169c4ef59085cf561997f420aa632944fe6c84aa3843d73e0ee02a44f0448 WatchSource:0}: Error finding container e6f169c4ef59085cf561997f420aa632944fe6c84aa3843d73e0ee02a44f0448: Status 404 returned error can't find the container with id e6f169c4ef59085cf561997f420aa632944fe6c84aa3843d73e0ee02a44f0448 Jul 22 11:50:09.592560 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:09.592532 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5071ae6_69e3_42fb_9b62_e35c5e89017f.slice/crio-d7081e8eb80b88d4f9025f1d71a823d4c6843078a4e4835322b1812c4393c4dd WatchSource:0}: Error finding container d7081e8eb80b88d4f9025f1d71a823d4c6843078a4e4835322b1812c4393c4dd: Status 404 returned error can't find the container with id d7081e8eb80b88d4f9025f1d71a823d4c6843078a4e4835322b1812c4393c4dd Jul 22 11:50:09.593521 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:09.593496 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea87c46_b392_4a6d_97cb_116c1d213d84.slice/crio-dc970d6d8eda70055000ed0ed5ec928dbe57470784e95f25a2b27891119bf5a3 WatchSource:0}: Error finding container dc970d6d8eda70055000ed0ed5ec928dbe57470784e95f25a2b27891119bf5a3: Status 404 returned error can't find the container with id dc970d6d8eda70055000ed0ed5ec928dbe57470784e95f25a2b27891119bf5a3 Jul 22 11:50:09.594094 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:09.594038 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be84eee_fb05_4086_9c78_d70bf3a3f00e.slice/crio-adb56aed4c4250a390a9bff5120f6ef608cb13be806227950eff4f28fa1ddfa9 WatchSource:0}: Error finding container adb56aed4c4250a390a9bff5120f6ef608cb13be806227950eff4f28fa1ddfa9: Status 404 returned error can't find the container with id adb56aed4c4250a390a9bff5120f6ef608cb13be806227950eff4f28fa1ddfa9 Jul 22 11:50:09.613400 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.613378 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.613514 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.613431 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:09.613553 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.613532 2534 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:50:09.613595 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.613574 2534 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:09.613595 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.613588 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:10.613572267 +0000 UTC m=+38.939419513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : secret "router-metrics-certs-default" not found Jul 22 11:50:09.613660 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.613612 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls podName:ba13c544-b6f5-4b80-92c8-3f672a89de5e nodeName:}" failed. No retries permitted until 2026-07-22 11:50:10.613600948 +0000 UTC m=+38.939448190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-brh7n" (UID: "ba13c544-b6f5-4b80-92c8-3f672a89de5e") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:09.613660 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.613625 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf29f\" (UniqueName: \"kubernetes.io/projected/3d00f959-4884-48cb-a81f-db9a8b81645c-kube-api-access-lf29f\") pod \"network-check-source-7bc5b6bf85-7vrx2\" (UID: \"3d00f959-4884-48cb-a81f-db9a8b81645c\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" Jul 22 11:50:09.613727 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.613660 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:09.613771 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.613727 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:10.613720982 +0000 UTC m=+38.939568223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : configmap references non-existent config key: service-ca.crt Jul 22 11:50:09.628623 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.628593 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf29f\" (UniqueName: \"kubernetes.io/projected/3d00f959-4884-48cb-a81f-db9a8b81645c-kube-api-access-lf29f\") pod \"network-check-source-7bc5b6bf85-7vrx2\" (UID: \"3d00f959-4884-48cb-a81f-db9a8b81645c\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" Jul 22 11:50:09.715004 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.714962 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:09.715164 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.715113 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 11:50:09.715239 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.715176 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls podName:a3b8eaf6-c269-473e-ad55-a76a50ad06e9 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:10.715161119 +0000 UTC m=+39.041008360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-r8f4m" (UID: "a3b8eaf6-c269-473e-ad55-a76a50ad06e9") : secret "samples-operator-tls" not found Jul 22 11:50:09.735694 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.735670 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" Jul 22 11:50:09.816254 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.816223 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:09.816397 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.816311 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:09.816397 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.816385 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:50:09.816494 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.816462 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls podName:50f09e4c-b2c9-40e7-a509-278f3b5fcb63 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:11.816440877 +0000 UTC m=+40.142288134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls") pod "dns-default-k494z" (UID: "50f09e4c-b2c9-40e7-a509-278f3b5fcb63") : secret "dns-default-metrics-tls" not found Jul 22 11:50:09.816494 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.816460 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 11:50:09.816494 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.816482 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-578cbd896f-dfpt2: secret "image-registry-tls" not found Jul 22 11:50:09.816635 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:09.816533 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls podName:6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a nodeName:}" failed. No retries permitted until 2026-07-22 11:50:11.81651639 +0000 UTC m=+40.142363634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls") pod "image-registry-578cbd896f-dfpt2" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a") : secret "image-registry-tls" not found Jul 22 11:50:09.874469 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:09.874439 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2"] Jul 22 11:50:09.875587 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:09.875563 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d00f959_4884_48cb_a81f_db9a8b81645c.slice/crio-05ef99805b721005661fadc9d1223f7ee216a23947340e430b8a89f3ffa89541 WatchSource:0}: Error finding container 05ef99805b721005661fadc9d1223f7ee216a23947340e430b8a89f3ffa89541: Status 404 returned error can't find the container with id 05ef99805b721005661fadc9d1223f7ee216a23947340e430b8a89f3ffa89541 Jul 22 11:50:10.322810 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.322709 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wm925" event={"ID":"05e907e6-7648-4881-9d40-a4ffffd9d2c5","Type":"ContainerStarted","Data":"4ba3f081d30825ff8b2ad688105fed5daab971abd5ca94ddcad360fa01e19888"} Jul 22 11:50:10.326327 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.326235 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" event={"ID":"a5071ae6-69e3-42fb-9b62-e35c5e89017f","Type":"ContainerStarted","Data":"d7081e8eb80b88d4f9025f1d71a823d4c6843078a4e4835322b1812c4393c4dd"} Jul 22 11:50:10.328982 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.328919 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" event={"ID":"2be84eee-fb05-4086-9c78-d70bf3a3f00e","Type":"ContainerStarted","Data":"adb56aed4c4250a390a9bff5120f6ef608cb13be806227950eff4f28fa1ddfa9"} Jul 22 11:50:10.330406 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.330313 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" event={"ID":"dea87c46-b392-4a6d-97cb-116c1d213d84","Type":"ContainerStarted","Data":"dc970d6d8eda70055000ed0ed5ec928dbe57470784e95f25a2b27891119bf5a3"} Jul 22 11:50:10.332152 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.332125 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-656k7" event={"ID":"ecf76870-fbfb-4e3f-80ad-7c83e7540f88","Type":"ContainerStarted","Data":"e45f62c7ddd6056a8919fdb801fda64b9f565ddf9356ec4e3af2ca210bbeed09"} Jul 22 11:50:10.337206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.337178 2534 generic.go:356] "Generic (PLEG): container finished" podID="0f24d5f9-8caa-4d15-974f-7dd17f4cd111" containerID="4f3a96555c7d8a608f657bb5dc1621dee615b1e9f7775c478030e1aed6d08faf" exitCode=0 Jul 22 11:50:10.337314 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.337255 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerDied","Data":"4f3a96555c7d8a608f657bb5dc1621dee615b1e9f7775c478030e1aed6d08faf"} Jul 22 11:50:10.342724 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.342661 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" event={"ID":"3d00f959-4884-48cb-a81f-db9a8b81645c","Type":"ContainerStarted","Data":"05ef99805b721005661fadc9d1223f7ee216a23947340e430b8a89f3ffa89541"} Jul 22 11:50:10.346646 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.346621 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" event={"ID":"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8","Type":"ContainerStarted","Data":"e6f169c4ef59085cf561997f420aa632944fe6c84aa3843d73e0ee02a44f0448"} Jul 22 11:50:10.624342 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.624051 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:10.624507 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.624238 2534 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:50:10.624507 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.624421 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:10.624507 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.624485 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:12.624462724 +0000 UTC m=+40.950309969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : secret "router-metrics-certs-default" not found Jul 22 11:50:10.624680 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.624536 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:10.624680 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.624546 2534 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:10.624680 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.624599 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls podName:ba13c544-b6f5-4b80-92c8-3f672a89de5e nodeName:}" failed. No retries permitted until 2026-07-22 11:50:12.624583025 +0000 UTC m=+40.950430283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-brh7n" (UID: "ba13c544-b6f5-4b80-92c8-3f672a89de5e") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:10.624680 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.624647 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:12.62463472 +0000 UTC m=+40.950481965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : configmap references non-existent config key: service-ca.crt Jul 22 11:50:10.726565 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:10.726517 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:10.726954 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.726762 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 11:50:10.726954 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:10.726856 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls podName:a3b8eaf6-c269-473e-ad55-a76a50ad06e9 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:12.726836186 +0000 UTC m=+41.052683432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-r8f4m" (UID: "a3b8eaf6-c269-473e-ad55-a76a50ad06e9") : secret "samples-operator-tls" not found Jul 22 11:50:11.369614 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:11.369415 2534 generic.go:356] "Generic (PLEG): container finished" podID="0f24d5f9-8caa-4d15-974f-7dd17f4cd111" containerID="23279c57d1084a21ae21af0557dde2915fa4f6f8cec8b1b1a73a4b0cc8284be1" exitCode=0 Jul 22 11:50:11.369614 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:11.369499 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerDied","Data":"23279c57d1084a21ae21af0557dde2915fa4f6f8cec8b1b1a73a4b0cc8284be1"} Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:11.838484 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:11.838592 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:11.838760 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:11.838775 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-578cbd896f-dfpt2: secret "image-registry-tls" not found Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:11.838848 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls podName:6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a nodeName:}" failed. No retries permitted until 2026-07-22 11:50:15.838828942 +0000 UTC m=+44.164676200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls") pod "image-registry-578cbd896f-dfpt2" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a") : secret "image-registry-tls" not found Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:11.839273 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:50:11.839357 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:11.839322 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls podName:50f09e4c-b2c9-40e7-a509-278f3b5fcb63 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:15.839306877 +0000 UTC m=+44.165154125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls") pod "dns-default-k494z" (UID: "50f09e4c-b2c9-40e7-a509-278f3b5fcb63") : secret "dns-default-metrics-tls" not found Jul 22 11:50:12.379679 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:12.379638 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" event={"ID":"0f24d5f9-8caa-4d15-974f-7dd17f4cd111","Type":"ContainerStarted","Data":"e55bbc81ff46934c82b8ede85889d3cdd8417bd6142697527bbd249d2fc125e8"} Jul 22 11:50:12.647038 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:12.646997 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:12.647214 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:12.647098 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:12.647214 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:12.647162 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:12.647214 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.647171 2534 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:50:12.647429 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.647256 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:16.647232737 +0000 UTC m=+44.973079989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : secret "router-metrics-certs-default" not found Jul 22 11:50:12.647429 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.647369 2534 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:12.647429 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.647418 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls podName:ba13c544-b6f5-4b80-92c8-3f672a89de5e nodeName:}" failed. No retries permitted until 2026-07-22 11:50:16.647404222 +0000 UTC m=+44.973251489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-brh7n" (UID: "ba13c544-b6f5-4b80-92c8-3f672a89de5e") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:12.647570 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.647492 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:16.647481524 +0000 UTC m=+44.973328783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : configmap references non-existent config key: service-ca.crt Jul 22 11:50:12.748572 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:12.748533 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:12.748841 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.748822 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 11:50:12.748945 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:12.748904 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls podName:a3b8eaf6-c269-473e-ad55-a76a50ad06e9 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:16.748873961 +0000 UTC m=+45.074721208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-r8f4m" (UID: "a3b8eaf6-c269-473e-ad55-a76a50ad06e9") : secret "samples-operator-tls" not found Jul 22 11:50:15.876827 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:15.876775 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:15.877267 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:15.876879 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:15.877267 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:15.876971 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:50:15.877267 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:15.877017 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 11:50:15.877267 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:15.877034 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-578cbd896f-dfpt2: secret "image-registry-tls" not found Jul 22 11:50:15.877267 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:15.877035 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls podName:50f09e4c-b2c9-40e7-a509-278f3b5fcb63 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:23.87701957 +0000 UTC m=+52.202866812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls") pod "dns-default-k494z" (UID: "50f09e4c-b2c9-40e7-a509-278f3b5fcb63") : secret "dns-default-metrics-tls" not found Jul 22 11:50:15.877267 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:15.877088 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls podName:6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a nodeName:}" failed. No retries permitted until 2026-07-22 11:50:23.877071476 +0000 UTC m=+52.202918717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls") pod "image-registry-578cbd896f-dfpt2" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a") : secret "image-registry-tls" not found Jul 22 11:50:16.685037 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:16.684997 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:16.685227 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:16.685086 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:16.685227 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.685162 2534 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:16.685227 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.685226 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls podName:ba13c544-b6f5-4b80-92c8-3f672a89de5e nodeName:}" failed. No retries permitted until 2026-07-22 11:50:24.685208649 +0000 UTC m=+53.011055891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-brh7n" (UID: "ba13c544-b6f5-4b80-92c8-3f672a89de5e") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:16.685373 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:16.685253 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:16.685373 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.685274 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:24.685255578 +0000 UTC m=+53.011102825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : configmap references non-existent config key: service-ca.crt Jul 22 11:50:16.685373 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.685319 2534 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:50:16.685373 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.685351 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:24.685342338 +0000 UTC m=+53.011189580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : secret "router-metrics-certs-default" not found Jul 22 11:50:16.786132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:16.786099 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:16.786287 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.786246 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 11:50:16.786327 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:16.786313 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls podName:a3b8eaf6-c269-473e-ad55-a76a50ad06e9 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:24.78629848 +0000 UTC m=+53.112145722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-r8f4m" (UID: "a3b8eaf6-c269-473e-ad55-a76a50ad06e9") : secret "samples-operator-tls" not found Jul 22 11:50:18.398571 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.398136 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-656k7" event={"ID":"ecf76870-fbfb-4e3f-80ad-7c83e7540f88","Type":"ContainerStarted","Data":"60a92a58ebfd018ff470282e6fc4b18b45fc80f8318979edb4b3203f5c850264"} Jul 22 11:50:18.399760 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.399726 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" event={"ID":"3d00f959-4884-48cb-a81f-db9a8b81645c","Type":"ContainerStarted","Data":"2a57071331048cedf8e712309d66020b6f1fecad9e8115df9d0f4d4ec53495ed"} Jul 22 11:50:18.401577 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.401553 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" event={"ID":"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8","Type":"ContainerStarted","Data":"991f52f9552b94eb0785dd2ef9735712a6caef834f2b17b22e3b94bc47ca2b44"} Jul 22 11:50:18.403092 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.403066 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wm925" event={"ID":"05e907e6-7648-4881-9d40-a4ffffd9d2c5","Type":"ContainerStarted","Data":"8bb84c05e6d123a4e2852396e3d26e48743769af32c144c23a3de3e0fade2ed1"} Jul 22 11:50:18.404713 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.404669 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" event={"ID":"a5071ae6-69e3-42fb-9b62-e35c5e89017f","Type":"ContainerStarted","Data":"ced2a3f1b5c6a5348d05a402237c65d6cdb15c7e48133dd90378a42f242b38e6"} Jul 22 11:50:18.406173 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.406156 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/0.log" Jul 22 11:50:18.406259 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.406188 2534 generic.go:356] "Generic (PLEG): container finished" podID="2be84eee-fb05-4086-9c78-d70bf3a3f00e" containerID="08065db413d536986cf1b94eebb4003bbd6f402599c264befbe751633f95964e" exitCode=255 Jul 22 11:50:18.406394 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.406350 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" event={"ID":"2be84eee-fb05-4086-9c78-d70bf3a3f00e","Type":"ContainerDied","Data":"08065db413d536986cf1b94eebb4003bbd6f402599c264befbe751633f95964e"} Jul 22 11:50:18.406454 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.406439 2534 scope.go:117] "RemoveContainer" containerID="08065db413d536986cf1b94eebb4003bbd6f402599c264befbe751633f95964e" Jul 22 11:50:18.408239 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.408143 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" event={"ID":"dea87c46-b392-4a6d-97cb-116c1d213d84","Type":"ContainerStarted","Data":"b28a78c74bdb24211da3ea05055dd4ce5e9eca8fb4ea97e56d8b75836c80b809"} Jul 22 11:50:18.430957 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.430904 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-656k7" podStartSLOduration=2.543878946 podStartE2EDuration="10.430882266s" totalImagesPullingTime="7.88700332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:18.430494097 +0000 UTC m=+46.756341362" watchObservedRunningTime="2026-07-22 11:50:18.430882266 +0000 UTC m=+46.756729534" Jul 22 11:50:18.431517 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.431484 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q2zv9" podStartSLOduration=14.319230874 podStartE2EDuration="45.431474703s" totalImagesPullingTime="31.112243829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:12.433198007 +0000 UTC m=+40.759045271" watchObservedRunningTime="2026-07-22 11:50:18.431474703 +0000 UTC m=+46.757321967" Jul 22 11:50:18.458553 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.458501 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" podStartSLOduration=2.667920482 podStartE2EDuration="10.458480913s" totalImagesPullingTime="7.790560431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:18.456643127 +0000 UTC m=+46.782490393" watchObservedRunningTime="2026-07-22 11:50:18.458480913 +0000 UTC m=+46.784328159" Jul 22 11:50:18.509227 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.509175 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7vrx2" podStartSLOduration=1.99127386 podStartE2EDuration="9.5091587s" totalImagesPullingTime="7.51788484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:18.484770313 +0000 UTC m=+46.810617579" watchObservedRunningTime="2026-07-22 11:50:18.5091587 +0000 UTC m=+46.835005964" Jul 22 11:50:18.509546 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.509521 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wm925" podStartSLOduration=2.6219734409999997 podStartE2EDuration="10.509514721s" totalImagesPullingTime="7.88754128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:18.508442146 +0000 UTC m=+46.834289410" watchObservedRunningTime="2026-07-22 11:50:18.509514721 +0000 UTC m=+46.835361976" Jul 22 11:50:18.531745 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.530033 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x6bsw" podStartSLOduration=2.8628058899999997 podStartE2EDuration="10.530014144s" totalImagesPullingTime="7.667208254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:18.528643805 +0000 UTC m=+46.854491076" watchObservedRunningTime="2026-07-22 11:50:18.530014144 +0000 UTC m=+46.855861410" Jul 22 11:50:18.590147 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:18.590065 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" podStartSLOduration=2.800147281 podStartE2EDuration="10.590024874s" totalImagesPullingTime="7.789877593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:18.589753498 +0000 UTC m=+46.915600760" watchObservedRunningTime="2026-07-22 11:50:18.590024874 +0000 UTC m=+46.915872134" Jul 22 11:50:19.002223 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.002176 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-787pj"] Jul 22 11:50:19.024507 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.024479 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.029252 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.029181 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 11:50:19.029701 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.029679 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8jdv6\"" Jul 22 11:50:19.030116 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.030094 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 11:50:19.030785 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.030761 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-787pj"] Jul 22 11:50:19.109652 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.109619 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3ea5425-46df-4c5f-993e-0e82b7417cb8-data-volume\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.109843 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.109662 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3ea5425-46df-4c5f-993e-0e82b7417cb8-crio-socket\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.109843 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.109690 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.109843 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.109773 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpbfh\" (UniqueName: \"kubernetes.io/projected/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-api-access-bpbfh\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.109958 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.109867 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211035 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211003 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211087 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3ea5425-46df-4c5f-993e-0e82b7417cb8-data-volume\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211113 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3ea5425-46df-4c5f-993e-0e82b7417cb8-crio-socket\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211137 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211163 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpbfh\" (UniqueName: \"kubernetes.io/projected/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-api-access-bpbfh\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211207 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.211177 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:50:19.211385 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.211246 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:50:19.211385 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.211258 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:19.711237711 +0000 UTC m=+48.037084967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : secret "insights-runtime-extractor-tls" not found Jul 22 11:50:19.211385 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.211304 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:19.711287034 +0000 UTC m=+48.037134275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : configmap "kube-rbac-proxy" not found Jul 22 11:50:19.211385 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211355 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3ea5425-46df-4c5f-993e-0e82b7417cb8-crio-socket\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.211528 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.211449 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3ea5425-46df-4c5f-993e-0e82b7417cb8-data-volume\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.227479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.227453 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpbfh\" (UniqueName: \"kubernetes.io/projected/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-api-access-bpbfh\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.315923 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.315835 2534 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:19.316068 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.315942 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:19.412855 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.412829 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:50:19.413291 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.413245 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/0.log" Jul 22 11:50:19.413347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.413285 2534 generic.go:356] "Generic (PLEG): container finished" podID="2be84eee-fb05-4086-9c78-d70bf3a3f00e" containerID="dd7ab752493625a6a4b81fcbae314a37c8386cc416b2ff588f853d9a790c8498" exitCode=255 Jul 22 11:50:19.413420 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.413388 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" event={"ID":"2be84eee-fb05-4086-9c78-d70bf3a3f00e","Type":"ContainerDied","Data":"dd7ab752493625a6a4b81fcbae314a37c8386cc416b2ff588f853d9a790c8498"} Jul 22 11:50:19.413460 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.413453 2534 scope.go:117] "RemoveContainer" containerID="08065db413d536986cf1b94eebb4003bbd6f402599c264befbe751633f95964e" Jul 22 11:50:19.413589 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.413572 2534 scope.go:117] "RemoveContainer" containerID="dd7ab752493625a6a4b81fcbae314a37c8386cc416b2ff588f853d9a790c8498" Jul 22 11:50:19.413816 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.413766 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-74m8f_openshift-console-operator(2be84eee-fb05-4086-9c78-d70bf3a3f00e)\"" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" podUID="2be84eee-fb05-4086-9c78-d70bf3a3f00e" Jul 22 11:50:19.716184 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.716151 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.716346 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:19.716245 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:19.716346 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.716282 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:50:19.716424 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.716362 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:20.716342222 +0000 UTC m=+49.042189477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : configmap "kube-rbac-proxy" not found Jul 22 11:50:19.716424 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.716389 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:50:19.716489 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:19.716448 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:20.716433946 +0000 UTC m=+49.042281208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : secret "insights-runtime-extractor-tls" not found Jul 22 11:50:20.420510 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:20.420483 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:50:20.420963 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:20.420877 2534 scope.go:117] "RemoveContainer" containerID="dd7ab752493625a6a4b81fcbae314a37c8386cc416b2ff588f853d9a790c8498" Jul 22 11:50:20.421104 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:20.421082 2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-74m8f_openshift-console-operator(2be84eee-fb05-4086-9c78-d70bf3a3f00e)\"" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" podUID="2be84eee-fb05-4086-9c78-d70bf3a3f00e" Jul 22 11:50:20.592911 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:20.592873 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngkzd_03f2645b-2fc8-4bef-9bec-128198246ad6/dns-node-resolver/0.log" Jul 22 11:50:20.724335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:20.724261 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:20.724479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:20.724341 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:20.724479 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:20.724389 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:50:20.724479 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:20.724438 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:50:20.724586 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:20.724489 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:22.724469714 +0000 UTC m=+51.050316962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : configmap "kube-rbac-proxy" not found Jul 22 11:50:20.724586 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:20.724505 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:22.724498759 +0000 UTC m=+51.050346001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : secret "insights-runtime-extractor-tls" not found Jul 22 11:50:21.404877 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:21.404853 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4c6jg_1fdb909c-8893-43f5-ba58-a8f774076194/node-ca/0.log" Jul 22 11:50:22.398134 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:22.398098 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wm925_05e907e6-7648-4881-9d40-a4ffffd9d2c5/serve-healthcheck-canary/0.log" Jul 22 11:50:22.742053 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:22.741954 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:22.742205 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:22.742105 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:22.742205 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:22.742114 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:50:22.742205 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:22.742175 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:26.742158667 +0000 UTC m=+55.068005922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : secret "insights-runtime-extractor-tls" not found Jul 22 11:50:22.742350 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:22.742216 2534 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:50:22.742350 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:22.742280 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:26.742266026 +0000 UTC m=+55.068113269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : configmap "kube-rbac-proxy" not found Jul 22 11:50:23.951284 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:23.951246 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:23.951651 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:23.951318 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:23.951651 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:23.951409 2534 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:50:23.951651 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:23.951474 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls podName:50f09e4c-b2c9-40e7-a509-278f3b5fcb63 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:39.951458549 +0000 UTC m=+68.277305791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls") pod "dns-default-k494z" (UID: "50f09e4c-b2c9-40e7-a509-278f3b5fcb63") : secret "dns-default-metrics-tls" not found Jul 22 11:50:23.951651 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:23.951475 2534 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 11:50:23.951651 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:23.951494 2534 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-578cbd896f-dfpt2: secret "image-registry-tls" not found Jul 22 11:50:23.951651 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:23.951540 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls podName:6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a nodeName:}" failed. No retries permitted until 2026-07-22 11:50:39.951526821 +0000 UTC m=+68.277374081 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls") pod "image-registry-578cbd896f-dfpt2" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a") : secret "image-registry-tls" not found Jul 22 11:50:24.756073 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:24.756016 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:24.756271 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:24.756100 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:24.756271 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:24.756143 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:24.756271 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.756180 2534 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:50:24.756271 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.756243 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:40.756227263 +0000 UTC m=+69.082074505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : secret "router-metrics-certs-default" not found Jul 22 11:50:24.756271 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.756248 2534 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:24.756459 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.756280 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle podName:8b58a72c-7ded-4849-a1fb-df32c70b6081 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:40.75626607 +0000 UTC m=+69.082113330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle") pod "router-default-79ccd9f656-4kgpj" (UID: "8b58a72c-7ded-4849-a1fb-df32c70b6081") : configmap references non-existent config key: service-ca.crt Jul 22 11:50:24.756459 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.756298 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls podName:ba13c544-b6f5-4b80-92c8-3f672a89de5e nodeName:}" failed. No retries permitted until 2026-07-22 11:50:40.756288667 +0000 UTC m=+69.082135913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-brh7n" (UID: "ba13c544-b6f5-4b80-92c8-3f672a89de5e") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:50:24.857404 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:24.857367 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:24.857596 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.857546 2534 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 11:50:24.857665 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:24.857627 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls podName:a3b8eaf6-c269-473e-ad55-a76a50ad06e9 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:40.857606123 +0000 UTC m=+69.183453389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-r8f4m" (UID: "a3b8eaf6-c269-473e-ad55-a76a50ad06e9") : secret "samples-operator-tls" not found Jul 22 11:50:26.774648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:26.774612 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:26.775244 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:26.774714 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:26.775244 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:26.774888 2534 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:50:26.775244 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:50:26.774968 2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls podName:b3ea5425-46df-4c5f-993e-0e82b7417cb8 nodeName:}" failed. No retries permitted until 2026-07-22 11:50:34.774946603 +0000 UTC m=+63.100793846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-787pj" (UID: "b3ea5425-46df-4c5f-993e-0e82b7417cb8") : secret "insights-runtime-extractor-tls" not found Jul 22 11:50:26.775356 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:26.775329 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3ea5425-46df-4c5f-993e-0e82b7417cb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:29.315547 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.315503 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:29.316092 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.315566 2534 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:29.316092 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.316011 2534 scope.go:117] "RemoveContainer" containerID="dd7ab752493625a6a4b81fcbae314a37c8386cc416b2ff588f853d9a790c8498" Jul 22 11:50:29.451363 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.451331 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:50:29.451526 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.451425 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" event={"ID":"2be84eee-fb05-4086-9c78-d70bf3a3f00e","Type":"ContainerStarted","Data":"19a1528d9e0eed504188d87a833e232d0f0699bcc65fa08a4f3e082fe34a0848"} Jul 22 11:50:29.451788 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.451766 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:29.471246 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.471188 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" podStartSLOduration=13.68286695 podStartE2EDuration="21.47117389s" totalImagesPullingTime="7.78830694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:29.470403384 +0000 UTC m=+57.796250647" watchObservedRunningTime="2026-07-22 11:50:29.47117389 +0000 UTC m=+57.797021153" Jul 22 11:50:29.755542 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:29.755514 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-74m8f" Jul 22 11:50:34.299446 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:34.299414 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2g487" Jul 22 11:50:34.844591 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:34.844544 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:34.847046 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:34.847010 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3ea5425-46df-4c5f-993e-0e82b7417cb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-787pj\" (UID: \"b3ea5425-46df-4c5f-993e-0e82b7417cb8\") " pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:34.938069 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:34.938042 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8jdv6\"" Jul 22 11:50:34.945242 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:34.945225 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-787pj" Jul 22 11:50:35.101151 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:35.101082 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-787pj"] Jul 22 11:50:35.104340 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:35.104315 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ea5425_46df_4c5f_993e_0e82b7417cb8.slice/crio-a57569e88301a93a3ccad5b22c3ae79dcd4d7a2faf3d633e1740eadbcdc3bab1 WatchSource:0}: Error finding container a57569e88301a93a3ccad5b22c3ae79dcd4d7a2faf3d633e1740eadbcdc3bab1: Status 404 returned error can't find the container with id a57569e88301a93a3ccad5b22c3ae79dcd4d7a2faf3d633e1740eadbcdc3bab1 Jul 22 11:50:35.471815 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:35.471755 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-787pj" event={"ID":"b3ea5425-46df-4c5f-993e-0e82b7417cb8","Type":"ContainerStarted","Data":"a57569e88301a93a3ccad5b22c3ae79dcd4d7a2faf3d633e1740eadbcdc3bab1"} Jul 22 11:50:37.773132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.773091 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:37.776431 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.776401 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 11:50:37.786860 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.786827 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80dd74e9-d654-4fa1-98bf-2bcf2395da42-metrics-certs\") pod \"network-metrics-daemon-sw68n\" (UID: \"80dd74e9-d654-4fa1-98bf-2bcf2395da42\") " pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:37.812836 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.812787 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mjncz\"" Jul 22 11:50:37.820504 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.820474 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sw68n" Jul 22 11:50:37.874658 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.874020 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:37.874658 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.874122 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:37.877508 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.877481 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 11:50:37.877698 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.877672 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d62f4\" (UniqueName: \"kubernetes.io/projected/94604095-766a-474c-9742-261c38fe2645-kube-api-access-d62f4\") pod \"network-check-target-9gbj2\" (UID: \"94604095-766a-474c-9742-261c38fe2645\") " pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:37.887307 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.887283 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5ae2be-b7bd-4758-a311-9104121792f1-original-pull-secret\") pod \"global-pull-secret-syncer-dkgxt\" (UID: \"ea5ae2be-b7bd-4758-a311-9104121792f1\") " pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:37.925919 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:37.925891 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkgxt" Jul 22 11:50:38.120484 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:38.120409 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b6dpz\"" Jul 22 11:50:38.128068 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:38.128047 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:39.992618 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:39.992448 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:39.992618 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:39.992535 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:39.995074 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:39.995044 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50f09e4c-b2c9-40e7-a509-278f3b5fcb63-metrics-tls\") pod \"dns-default-k494z\" (UID: \"50f09e4c-b2c9-40e7-a509-278f3b5fcb63\") " pod="openshift-dns/dns-default-k494z" Jul 22 11:50:39.997207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:39.997172 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"image-registry-578cbd896f-dfpt2\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:40.206670 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.206510 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f7k7c\"" Jul 22 11:50:40.214159 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.214132 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:40.228586 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.228554 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2xm6s\"" Jul 22 11:50:40.236077 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.236051 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k494z" Jul 22 11:50:40.368131 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.368099 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sw68n"] Jul 22 11:50:40.376676 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:40.376644 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80dd74e9_d654_4fa1_98bf_2bcf2395da42.slice/crio-684b3a2637ffdcc987903c51c6851025dc2810f7d153472fd7969d02c1ff7966 WatchSource:0}: Error finding container 684b3a2637ffdcc987903c51c6851025dc2810f7d153472fd7969d02c1ff7966: Status 404 returned error can't find the container with id 684b3a2637ffdcc987903c51c6851025dc2810f7d153472fd7969d02c1ff7966 Jul 22 11:50:40.396337 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.396269 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gbj2"] Jul 22 11:50:40.469861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.469686 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-578cbd896f-dfpt2"] Jul 22 11:50:40.475897 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.475871 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k494z"] Jul 22 11:50:40.488567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.488543 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sw68n" event={"ID":"80dd74e9-d654-4fa1-98bf-2bcf2395da42","Type":"ContainerStarted","Data":"684b3a2637ffdcc987903c51c6851025dc2810f7d153472fd7969d02c1ff7966"} Jul 22 11:50:40.596102 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:40.595997 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94604095_766a_474c_9742_261c38fe2645.slice/crio-c937245f2a058801db53ff13ab644732506714daae5c1c572083683d632d18f0 WatchSource:0}: Error finding container c937245f2a058801db53ff13ab644732506714daae5c1c572083683d632d18f0: Status 404 returned error can't find the container with id c937245f2a058801db53ff13ab644732506714daae5c1c572083683d632d18f0 Jul 22 11:50:40.598085 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.598052 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkgxt"] Jul 22 11:50:40.598203 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:40.598099 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d68bbc7_edbd_4e00_b07f_f6c33d7b5b5a.slice/crio-e11808760f4d3e4161ed1328686228c32d41e69e32141e1aa5c0d1ffebfabc38 WatchSource:0}: Error finding container e11808760f4d3e4161ed1328686228c32d41e69e32141e1aa5c0d1ffebfabc38: Status 404 returned error can't find the container with id e11808760f4d3e4161ed1328686228c32d41e69e32141e1aa5c0d1ffebfabc38 Jul 22 11:50:40.601625 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:40.601606 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5ae2be_b7bd_4758_a311_9104121792f1.slice/crio-2faec5d92686150c3e1ffae3a8b9b59fb8399bcc73695b6d89b3a974f7584148 WatchSource:0}: Error finding container 2faec5d92686150c3e1ffae3a8b9b59fb8399bcc73695b6d89b3a974f7584148: Status 404 returned error can't find the container with id 2faec5d92686150c3e1ffae3a8b9b59fb8399bcc73695b6d89b3a974f7584148 Jul 22 11:50:40.801581 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.801539 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:40.801739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.801589 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:40.801739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.801626 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:40.802246 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.802222 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b58a72c-7ded-4849-a1fb-df32c70b6081-service-ca-bundle\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:40.804211 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.804188 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b58a72c-7ded-4849-a1fb-df32c70b6081-metrics-certs\") pod \"router-default-79ccd9f656-4kgpj\" (UID: \"8b58a72c-7ded-4849-a1fb-df32c70b6081\") " pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:40.804320 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.804299 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba13c544-b6f5-4b80-92c8-3f672a89de5e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-brh7n\" (UID: \"ba13c544-b6f5-4b80-92c8-3f672a89de5e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:40.902508 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.902471 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:40.905411 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:40.905384 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b8eaf6-c269-473e-ad55-a76a50ad06e9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-r8f4m\" (UID: \"a3b8eaf6-c269-473e-ad55-a76a50ad06e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:41.017332 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.017291 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wvtm7\"" Jul 22 11:50:41.025599 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.025572 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" Jul 22 11:50:41.026738 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.026716 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vljkf\"" Jul 22 11:50:41.035531 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.035502 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:41.110744 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.110707 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jnbgt\"" Jul 22 11:50:41.118531 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.118286 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" Jul 22 11:50:41.189588 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.189477 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n"] Jul 22 11:50:41.214872 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.214842 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79ccd9f656-4kgpj"] Jul 22 11:50:41.494610 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.494521 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gbj2" event={"ID":"94604095-766a-474c-9742-261c38fe2645","Type":"ContainerStarted","Data":"359b798174b2356a99e8219bef0640d9e38edc1ec8635d25370a9898b65bcb6c"} Jul 22 11:50:41.494610 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.494590 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gbj2" event={"ID":"94604095-766a-474c-9742-261c38fe2645","Type":"ContainerStarted","Data":"c937245f2a058801db53ff13ab644732506714daae5c1c572083683d632d18f0"} Jul 22 11:50:41.494828 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.494650 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:50:41.496586 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.496522 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-787pj" event={"ID":"b3ea5425-46df-4c5f-993e-0e82b7417cb8","Type":"ContainerStarted","Data":"4c0584ff6ca31769a0c54a9def029ad9fe5f1d1ea0110e7158b93c8bb3dcc501"} Jul 22 11:50:41.498621 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.498556 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k494z" event={"ID":"50f09e4c-b2c9-40e7-a509-278f3b5fcb63","Type":"ContainerStarted","Data":"913fddda16f814b5174aa977d27c1813808170808df8b9724270edd7eeba39e5"} Jul 22 11:50:41.500926 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.500899 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkgxt" event={"ID":"ea5ae2be-b7bd-4758-a311-9104121792f1","Type":"ContainerStarted","Data":"2faec5d92686150c3e1ffae3a8b9b59fb8399bcc73695b6d89b3a974f7584148"} Jul 22 11:50:41.502760 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.502738 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" event={"ID":"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a","Type":"ContainerStarted","Data":"872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f"} Jul 22 11:50:41.502905 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.502777 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" event={"ID":"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a","Type":"ContainerStarted","Data":"e11808760f4d3e4161ed1328686228c32d41e69e32141e1aa5c0d1ffebfabc38"} Jul 22 11:50:41.503001 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.502910 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:50:41.508257 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:41.508230 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b58a72c_7ded_4849_a1fb_df32c70b6081.slice/crio-e9c2bf5ff114a3f2bb9059c3556ad48f6eaf748fc1a081195afee1091dd0ae97 WatchSource:0}: Error finding container e9c2bf5ff114a3f2bb9059c3556ad48f6eaf748fc1a081195afee1091dd0ae97: Status 404 returned error can't find the container with id e9c2bf5ff114a3f2bb9059c3556ad48f6eaf748fc1a081195afee1091dd0ae97 Jul 22 11:50:41.514042 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.513684 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9gbj2" podStartSLOduration=68.513664452 podStartE2EDuration="1m8.513664452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:41.512582259 +0000 UTC m=+69.838429523" watchObservedRunningTime="2026-07-22 11:50:41.513664452 +0000 UTC m=+69.839511713" Jul 22 11:50:41.534076 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.534016 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" podStartSLOduration=68.533994235 podStartE2EDuration="1m8.533994235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:41.533755213 +0000 UTC m=+69.859602478" watchObservedRunningTime="2026-07-22 11:50:41.533994235 +0000 UTC m=+69.859841500" Jul 22 11:50:41.680350 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:41.680321 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m"] Jul 22 11:50:42.511554 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.511451 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" event={"ID":"8b58a72c-7ded-4849-a1fb-df32c70b6081","Type":"ContainerStarted","Data":"f6f36f910ab73f22cbfecfc4c55c952874efe41b18e04428087fe1e9b5984fa1"} Jul 22 11:50:42.511554 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.511519 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" event={"ID":"8b58a72c-7ded-4849-a1fb-df32c70b6081","Type":"ContainerStarted","Data":"e9c2bf5ff114a3f2bb9059c3556ad48f6eaf748fc1a081195afee1091dd0ae97"} Jul 22 11:50:42.515340 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.514734 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sw68n" event={"ID":"80dd74e9-d654-4fa1-98bf-2bcf2395da42","Type":"ContainerStarted","Data":"24d10cfe5fce5207b8d0c2c4399ba2f46fca0fea35b558c7ab0a510d425a9fef"} Jul 22 11:50:42.515340 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.514811 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sw68n" event={"ID":"80dd74e9-d654-4fa1-98bf-2bcf2395da42","Type":"ContainerStarted","Data":"01e3b06ae8c23ee5b46d8ce3493fb754f76adc03e2848cdf135cb060809a01d5"} Jul 22 11:50:42.518814 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.518771 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" event={"ID":"a3b8eaf6-c269-473e-ad55-a76a50ad06e9","Type":"ContainerStarted","Data":"679ee82c8cfd688d7871025ccb811f31ae0dded57001c9f1de932fa9821c519b"} Jul 22 11:50:42.520372 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.520300 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" event={"ID":"ba13c544-b6f5-4b80-92c8-3f672a89de5e","Type":"ContainerStarted","Data":"439e1747d7f26d2c76ce0078cb1afb634c78286603af1ffce8d4438f93e25f9a"} Jul 22 11:50:42.545488 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.545423 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" podStartSLOduration=34.545401197 podStartE2EDuration="34.545401197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:42.543788109 +0000 UTC m=+70.869635375" watchObservedRunningTime="2026-07-22 11:50:42.545401197 +0000 UTC m=+70.871248442" Jul 22 11:50:42.571132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:42.571068 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sw68n" podStartSLOduration=68.255757383 podStartE2EDuration="1m9.57104403s" totalImagesPullingTime="1.315286647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:42.570555678 +0000 UTC m=+70.896402943" watchObservedRunningTime="2026-07-22 11:50:42.57104403 +0000 UTC m=+70.896891299" Jul 22 11:50:43.036158 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:43.036114 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:43.039172 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:43.038936 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:43.526780 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:43.526730 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:43.528811 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:43.528605 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79ccd9f656-4kgpj" Jul 22 11:50:48.004337 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.004302 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7484cf5f9b-h4jjw"] Jul 22 11:50:48.032564 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.032532 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-4d8kg"] Jul 22 11:50:48.032706 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.032673 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.041421 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041391 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 11:50:48.041559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041446 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 11:50:48.041559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041404 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 11:50:48.041777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041757 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sdcc8\"" Jul 22 11:50:48.041959 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041815 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 11:50:48.041959 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041859 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 11:50:48.041959 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.041873 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 11:50:48.042206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.042190 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 11:50:48.048492 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.048470 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc"] Jul 22 11:50:48.048583 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.048574 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:50:48.054422 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.051781 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7l49x\"" Jul 22 11:50:48.078987 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.078961 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc"] Jul 22 11:50:48.078987 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.078992 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7484cf5f9b-h4jjw"] Jul 22 11:50:48.079133 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.079006 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-4d8kg"] Jul 22 11:50:48.079133 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.079072 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.083645 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.083623 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 11:50:48.083769 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.083718 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-njd27\"" Jul 22 11:50:48.083857 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.083626 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 11:50:48.171784 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.171758 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-config\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.171913 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.171823 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-service-ca\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.171979 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.171932 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-oauth-config\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.171979 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.171964 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-serving-cert\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.172065 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.171990 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ed4af996-0621-404d-b718-a5f7336fa640-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-wxrfc\" (UID: \"ed4af996-0621-404d-b718-a5f7336fa640\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.172065 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.172028 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed4af996-0621-404d-b718-a5f7336fa640-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-wxrfc\" (UID: \"ed4af996-0621-404d-b718-a5f7336fa640\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.172065 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.172061 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfjtm\" (UniqueName: \"kubernetes.io/projected/eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2-kube-api-access-hfjtm\") pod \"downloads-5bc67dcd46-4d8kg\" (UID: \"eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2\") " pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:50:48.172157 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.172079 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-oauth-serving-cert\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.172157 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.172108 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxknb\" (UniqueName: \"kubernetes.io/projected/e86cfe83-2a62-4af5-8e12-8a5276de2a21-kube-api-access-xxknb\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273338 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-config\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273384 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-service-ca\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273451 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-oauth-config\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273492 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-serving-cert\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273518 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ed4af996-0621-404d-b718-a5f7336fa640-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-wxrfc\" (UID: \"ed4af996-0621-404d-b718-a5f7336fa640\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273552 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed4af996-0621-404d-b718-a5f7336fa640-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-wxrfc\" (UID: \"ed4af996-0621-404d-b718-a5f7336fa640\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273596 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfjtm\" (UniqueName: \"kubernetes.io/projected/eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2-kube-api-access-hfjtm\") pod \"downloads-5bc67dcd46-4d8kg\" (UID: \"eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2\") " pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273623 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-oauth-serving-cert\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.273651 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxknb\" (UniqueName: \"kubernetes.io/projected/e86cfe83-2a62-4af5-8e12-8a5276de2a21-kube-api-access-xxknb\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.274827 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-config\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.275435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.275386 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-service-ca\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.276857 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.276831 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-oauth-serving-cert\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.277535 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.277512 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ed4af996-0621-404d-b718-a5f7336fa640-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-wxrfc\" (UID: \"ed4af996-0621-404d-b718-a5f7336fa640\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.277885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.277866 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-oauth-config\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.278687 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.278647 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ed4af996-0621-404d-b718-a5f7336fa640-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-wxrfc\" (UID: \"ed4af996-0621-404d-b718-a5f7336fa640\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.279436 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.279415 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-serving-cert\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.295383 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.295318 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfjtm\" (UniqueName: \"kubernetes.io/projected/eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2-kube-api-access-hfjtm\") pod \"downloads-5bc67dcd46-4d8kg\" (UID: \"eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2\") " pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:50:48.295523 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.295512 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxknb\" (UniqueName: \"kubernetes.io/projected/e86cfe83-2a62-4af5-8e12-8a5276de2a21-kube-api-access-xxknb\") pod \"console-7484cf5f9b-h4jjw\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.345858 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.345822 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:48.359732 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.359700 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:50:48.388365 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.388317 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" Jul 22 11:50:48.563280 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.561618 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k494z" event={"ID":"50f09e4c-b2c9-40e7-a509-278f3b5fcb63","Type":"ContainerStarted","Data":"7c16c711bb68fd9017b8634e507bcf739fcf3f022b4527c3dbe1308064979a42"} Jul 22 11:50:48.563894 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.563863 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkgxt" event={"ID":"ea5ae2be-b7bd-4758-a311-9104121792f1","Type":"ContainerStarted","Data":"065ca3257832a5b0ddfa8a23f9fb29c733b85f5f8b58d7e547171ab3a28eefd3"} Jul 22 11:50:48.595861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.595743 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" event={"ID":"a3b8eaf6-c269-473e-ad55-a76a50ad06e9","Type":"ContainerStarted","Data":"b087b95e1dec7190f67d52ecc057a7c06f62f9f60521617fadcf38ff0f6efb58"} Jul 22 11:50:48.608687 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.607656 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" event={"ID":"ba13c544-b6f5-4b80-92c8-3f672a89de5e","Type":"ContainerStarted","Data":"54cbe8d0e9028919977abf3a76fd05cd89554f9f47e91eb0f531e19f3e95f24e"} Jul 22 11:50:48.664189 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.660985 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dkgxt" podStartSLOduration=68.202860336 podStartE2EDuration="1m15.660935741s" totalImagesPullingTime="7.458075405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:48.608338907 +0000 UTC m=+76.934186175" watchObservedRunningTime="2026-07-22 11:50:48.660935741 +0000 UTC m=+76.986783006" Jul 22 11:50:48.664189 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.662073 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-brh7n" podStartSLOduration=34.084743436 podStartE2EDuration="40.662057132s" totalImagesPullingTime="6.577313696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:48.66012373 +0000 UTC m=+76.985970995" watchObservedRunningTime="2026-07-22 11:50:48.662057132 +0000 UTC m=+76.987904398" Jul 22 11:50:48.704172 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.701992 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc"] Jul 22 11:50:48.712685 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:48.712076 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4af996_0621_404d_b718_a5f7336fa640.slice/crio-6235472c69feaa1da42df772eb6bf606ef84ab1e6dd7fcafdc2e647ee503a38b WatchSource:0}: Error finding container 6235472c69feaa1da42df772eb6bf606ef84ab1e6dd7fcafdc2e647ee503a38b: Status 404 returned error can't find the container with id 6235472c69feaa1da42df772eb6bf606ef84ab1e6dd7fcafdc2e647ee503a38b Jul 22 11:50:48.754024 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.753994 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7484cf5f9b-h4jjw"] Jul 22 11:50:48.757283 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:48.757255 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86cfe83_2a62_4af5_8e12_8a5276de2a21.slice/crio-4d9f84af26365d487fd93be00b4e11c23be12a876adde219f7e6aa14312897ca WatchSource:0}: Error finding container 4d9f84af26365d487fd93be00b4e11c23be12a876adde219f7e6aa14312897ca: Status 404 returned error can't find the container with id 4d9f84af26365d487fd93be00b4e11c23be12a876adde219f7e6aa14312897ca Jul 22 11:50:48.762084 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.762060 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-4d8kg"] Jul 22 11:50:48.764147 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:48.764121 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeedfa2bc_04ce_41e3_bfb7_0e6c3a5e3fe2.slice/crio-d48d11fd38928bb40d3ca01d7ff3016d2a70a707c2cdca8f5215d86cfad9131c WatchSource:0}: Error finding container d48d11fd38928bb40d3ca01d7ff3016d2a70a707c2cdca8f5215d86cfad9131c: Status 404 returned error can't find the container with id d48d11fd38928bb40d3ca01d7ff3016d2a70a707c2cdca8f5215d86cfad9131c Jul 22 11:50:48.981665 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:48.981578 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd"] Jul 22 11:50:49.016214 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.016185 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd"] Jul 22 11:50:49.016634 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.016286 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:49.021025 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.021002 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 11:50:49.021088 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.021048 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-w4s6r\"" Jul 22 11:50:49.180300 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.180265 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/878134b9-f138-4d1f-96cb-ad899a981140-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-k4lxd\" (UID: \"878134b9-f138-4d1f-96cb-ad899a981140\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:49.281203 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.281130 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/878134b9-f138-4d1f-96cb-ad899a981140-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-k4lxd\" (UID: \"878134b9-f138-4d1f-96cb-ad899a981140\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:49.284003 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.283976 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/878134b9-f138-4d1f-96cb-ad899a981140-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-k4lxd\" (UID: \"878134b9-f138-4d1f-96cb-ad899a981140\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:49.325848 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.325636 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:49.515014 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.514936 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd"] Jul 22 11:50:49.520855 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:50:49.520812 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878134b9_f138_4d1f_96cb_ad899a981140.slice/crio-558d03f3661d2825f1899afffa52e0d513561a2054704d48a1f610056f24db4f WatchSource:0}: Error finding container 558d03f3661d2825f1899afffa52e0d513561a2054704d48a1f610056f24db4f: Status 404 returned error can't find the container with id 558d03f3661d2825f1899afffa52e0d513561a2054704d48a1f610056f24db4f Jul 22 11:50:49.615726 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.615686 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" event={"ID":"a3b8eaf6-c269-473e-ad55-a76a50ad06e9","Type":"ContainerStarted","Data":"f3fab49f96d2014c2aba7db596c14de809e24572cd2f1f036e37fabb20de14c1"} Jul 22 11:50:49.618330 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.618299 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7484cf5f9b-h4jjw" event={"ID":"e86cfe83-2a62-4af5-8e12-8a5276de2a21","Type":"ContainerStarted","Data":"4d9f84af26365d487fd93be00b4e11c23be12a876adde219f7e6aa14312897ca"} Jul 22 11:50:49.620602 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.620547 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" event={"ID":"ed4af996-0621-404d-b718-a5f7336fa640","Type":"ContainerStarted","Data":"6235472c69feaa1da42df772eb6bf606ef84ab1e6dd7fcafdc2e647ee503a38b"} Jul 22 11:50:49.630077 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.629277 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k494z" event={"ID":"50f09e4c-b2c9-40e7-a509-278f3b5fcb63","Type":"ContainerStarted","Data":"c64632f74089b9298f09a75eb5ca5b312edcfe99fcbf32edad541879f84c7f20"} Jul 22 11:50:49.630077 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.629405 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k494z" Jul 22 11:50:49.632508 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.632476 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" event={"ID":"878134b9-f138-4d1f-96cb-ad899a981140","Type":"ContainerStarted","Data":"558d03f3661d2825f1899afffa52e0d513561a2054704d48a1f610056f24db4f"} Jul 22 11:50:49.634537 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.634509 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-4d8kg" event={"ID":"eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2","Type":"ContainerStarted","Data":"d48d11fd38928bb40d3ca01d7ff3016d2a70a707c2cdca8f5215d86cfad9131c"} Jul 22 11:50:49.636735 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.636698 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-787pj" event={"ID":"b3ea5425-46df-4c5f-993e-0e82b7417cb8","Type":"ContainerStarted","Data":"9be32f35b3dd48f3755ec4d22301f971ad584f029f8a58ed41dacf6ce528bea2"} Jul 22 11:50:49.672306 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:49.670430 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-r8f4m" podStartSLOduration=35.319682157 podStartE2EDuration="41.670405648s" totalImagesPullingTime="6.350723491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:49.644477264 +0000 UTC m=+77.970324520" watchObservedRunningTime="2026-07-22 11:50:49.670405648 +0000 UTC m=+77.996252913" Jul 22 11:50:51.650029 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:51.649985 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" event={"ID":"ed4af996-0621-404d-b718-a5f7336fa640","Type":"ContainerStarted","Data":"9b734166f9b88645813361644087036260c092a38fdf333b4443d740faf00f9c"} Jul 22 11:50:51.679189 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:51.679126 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wxrfc" podStartSLOduration=2.75425178 podStartE2EDuration="4.679105581s" totalImagesPullingTime="1.924853801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:51.677493603 +0000 UTC m=+80.003340870" watchObservedRunningTime="2026-07-22 11:50:51.679105581 +0000 UTC m=+80.004952842" Jul 22 11:50:51.679421 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:51.679391 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k494z" podStartSLOduration=36.370752117 podStartE2EDuration="43.679378321s" totalImagesPullingTime="7.308626204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:49.672254438 +0000 UTC m=+77.998101715" watchObservedRunningTime="2026-07-22 11:50:51.679378321 +0000 UTC m=+80.005225590" Jul 22 11:50:55.669609 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.669574 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" event={"ID":"878134b9-f138-4d1f-96cb-ad899a981140","Type":"ContainerStarted","Data":"ba82136cb572622d1846236b82869979e9196332dfb54e3b2f627a90543bd909"} Jul 22 11:50:55.670100 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.669745 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:55.672042 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.671986 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-787pj" event={"ID":"b3ea5425-46df-4c5f-993e-0e82b7417cb8","Type":"ContainerStarted","Data":"98e8c8664212c535c6aa86459ea70ac06c1957ad2f4a7a8c0d93a4f06535fd89"} Jul 22 11:50:55.673756 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.673730 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7484cf5f9b-h4jjw" event={"ID":"e86cfe83-2a62-4af5-8e12-8a5276de2a21","Type":"ContainerStarted","Data":"c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55"} Jul 22 11:50:55.675388 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.675367 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" Jul 22 11:50:55.690073 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.690017 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-k4lxd" podStartSLOduration=2.261933156 podStartE2EDuration="7.689998217s" totalImagesPullingTime="5.428065061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:55.687681123 +0000 UTC m=+84.013528391" watchObservedRunningTime="2026-07-22 11:50:55.689998217 +0000 UTC m=+84.015845484" Jul 22 11:50:55.717297 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.717241 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7484cf5f9b-h4jjw" podStartSLOduration=2.51618455 podStartE2EDuration="8.717224139s" totalImagesPullingTime="6.201039589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:55.717003958 +0000 UTC m=+84.042851227" watchObservedRunningTime="2026-07-22 11:50:55.717224139 +0000 UTC m=+84.043071402" Jul 22 11:50:55.756944 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:55.756885 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-787pj" podStartSLOduration=18.246375052 podStartE2EDuration="37.756867026s" totalImagesPullingTime="19.510491974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:50:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:50:55.754713633 +0000 UTC m=+84.080560898" watchObservedRunningTime="2026-07-22 11:50:55.756867026 +0000 UTC m=+84.082714290" Jul 22 11:50:58.347031 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:58.346996 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:58.347585 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:58.347071 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:58.352585 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:58.352558 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:58.692082 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:58.692054 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:50:59.646147 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:50:59.646111 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k494z" Jul 22 11:51:02.525709 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:02.525678 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:51:05.593457 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.593421 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gms99"] Jul 22 11:51:05.766663 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.766633 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.771264 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.771240 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p2zfp\"" Jul 22 11:51:05.771505 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.771487 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 11:51:05.771590 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.771528 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 11:51:05.771819 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.771787 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 11:51:05.772153 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.772134 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 11:51:05.816274 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816245 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-textfile\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816430 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816280 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-metrics-client-ca\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816430 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816309 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816430 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816365 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-sys\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816430 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816411 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-root\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816592 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816435 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-wtmp\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816592 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816465 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm6rk\" (UniqueName: \"kubernetes.io/projected/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-kube-api-access-nm6rk\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816592 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816539 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.816592 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.816564 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-tls\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917614 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-textfile\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917653 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-metrics-client-ca\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917677 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917706 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-sys\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917733 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-root\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917753 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-wtmp\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917778 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm6rk\" (UniqueName: \"kubernetes.io/projected/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-kube-api-access-nm6rk\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917849 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917865 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-root\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.917908 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.917881 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-tls\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.918224 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.918045 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-wtmp\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.918224 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.918129 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-sys\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.918564 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.918496 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.918694 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.918647 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-textfile\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.918941 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.918899 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-metrics-client-ca\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.920507 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.920487 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-tls\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.920605 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.920515 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:05.930111 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:05.930092 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm6rk\" (UniqueName: \"kubernetes.io/projected/abd5a4a0-46ee-441c-98fe-97dc1e00c6d2-kube-api-access-nm6rk\") pod \"node-exporter-gms99\" (UID: \"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2\") " pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:06.077933 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.077891 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gms99" Jul 22 11:51:06.366327 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:51:06.366294 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd5a4a0_46ee_441c_98fe_97dc1e00c6d2.slice/crio-622100ca6e53ee71bbe64eb91c0709a68f48d3dbdd05cba44c8a1d613244319f WatchSource:0}: Error finding container 622100ca6e53ee71bbe64eb91c0709a68f48d3dbdd05cba44c8a1d613244319f: Status 404 returned error can't find the container with id 622100ca6e53ee71bbe64eb91c0709a68f48d3dbdd05cba44c8a1d613244319f Jul 22 11:51:06.728023 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.727981 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-4d8kg" event={"ID":"eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2","Type":"ContainerStarted","Data":"84d2c5b8c5df0f13088ea33602582d0c24d15e27bf3a091b7f77c2561684ea7f"} Jul 22 11:51:06.728491 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.728312 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:51:06.729209 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.729186 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gms99" event={"ID":"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2","Type":"ContainerStarted","Data":"622100ca6e53ee71bbe64eb91c0709a68f48d3dbdd05cba44c8a1d613244319f"} Jul 22 11:51:06.729947 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.729920 2534 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-4d8kg container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" start-of-body= Jul 22 11:51:06.730056 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.729965 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-4d8kg" podUID="eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" Jul 22 11:51:06.774569 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:06.774509 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-4d8kg" podStartSLOduration=1.938310974 podStartE2EDuration="19.774487506s" totalImagesPullingTime="17.836176532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:51:06.774057179 +0000 UTC m=+95.099904436" watchObservedRunningTime="2026-07-22 11:51:06.774487506 +0000 UTC m=+95.100334770" Jul 22 11:51:07.747176 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:07.747148 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-4d8kg" Jul 22 11:51:08.740881 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:08.740843 2534 generic.go:356] "Generic (PLEG): container finished" podID="abd5a4a0-46ee-441c-98fe-97dc1e00c6d2" containerID="bcb5ed37883ad23ea9945817c81472529fb6291a9500271d86f7899ddd030443" exitCode=0 Jul 22 11:51:08.741082 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:08.740938 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gms99" event={"ID":"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2","Type":"ContainerDied","Data":"bcb5ed37883ad23ea9945817c81472529fb6291a9500271d86f7899ddd030443"} Jul 22 11:51:09.748193 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:09.748152 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gms99" event={"ID":"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2","Type":"ContainerStarted","Data":"c51acf08549a349c699b296fd986be8f1e6d03ff8883bf48c866630fe5a15c6d"} Jul 22 11:51:09.748629 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:09.748208 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gms99" event={"ID":"abd5a4a0-46ee-441c-98fe-97dc1e00c6d2","Type":"ContainerStarted","Data":"37c932370f37f7e739262751d3ba07b8b56013c30aabf7967f99ddde004b27e0"} Jul 22 11:51:09.772873 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:09.772822 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gms99" podStartSLOduration=3.2134341 podStartE2EDuration="4.772783255s" totalImagesPullingTime="1.438574281s" totalInitContainerRuntime="120.774874ms" isStatefulPod=true podCreationTimestamp="2026-07-22 11:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:51:09.771906881 +0000 UTC m=+98.097754147" watchObservedRunningTime="2026-07-22 11:51:09.772783255 +0000 UTC m=+98.098630520" Jul 22 11:51:12.414758 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:12.414717 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-578cbd896f-dfpt2"] Jul 22 11:51:12.524243 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:12.524147 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9gbj2" Jul 22 11:51:21.026383 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:21.026343 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7484cf5f9b-h4jjw"] Jul 22 11:51:23.805655 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:23.805622 2534 generic.go:356] "Generic (PLEG): container finished" podID="ecf76870-fbfb-4e3f-80ad-7c83e7540f88" containerID="60a92a58ebfd018ff470282e6fc4b18b45fc80f8318979edb4b3203f5c850264" exitCode=0 Jul 22 11:51:23.806126 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:23.805667 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-656k7" event={"ID":"ecf76870-fbfb-4e3f-80ad-7c83e7540f88","Type":"ContainerDied","Data":"60a92a58ebfd018ff470282e6fc4b18b45fc80f8318979edb4b3203f5c850264"} Jul 22 11:51:23.806126 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:23.806040 2534 scope.go:117] "RemoveContainer" containerID="60a92a58ebfd018ff470282e6fc4b18b45fc80f8318979edb4b3203f5c850264" Jul 22 11:51:24.810953 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:24.810917 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-656k7" event={"ID":"ecf76870-fbfb-4e3f-80ad-7c83e7540f88","Type":"ContainerStarted","Data":"3a673cc25b11146bf710b1666ad8556129971466dd44fda506416733463f01b1"} Jul 22 11:51:33.847337 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:33.847304 2534 generic.go:356] "Generic (PLEG): container finished" podID="0ba8d0cb-2b77-48d8-b186-b4152f82d6f8" containerID="991f52f9552b94eb0785dd2ef9735712a6caef834f2b17b22e3b94bc47ca2b44" exitCode=0 Jul 22 11:51:33.847819 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:33.847379 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" event={"ID":"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8","Type":"ContainerDied","Data":"991f52f9552b94eb0785dd2ef9735712a6caef834f2b17b22e3b94bc47ca2b44"} Jul 22 11:51:33.847819 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:33.847739 2534 scope.go:117] "RemoveContainer" containerID="991f52f9552b94eb0785dd2ef9735712a6caef834f2b17b22e3b94bc47ca2b44" Jul 22 11:51:33.848931 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:33.848910 2534 generic.go:356] "Generic (PLEG): container finished" podID="a5071ae6-69e3-42fb-9b62-e35c5e89017f" containerID="ced2a3f1b5c6a5348d05a402237c65d6cdb15c7e48133dd90378a42f242b38e6" exitCode=0 Jul 22 11:51:33.849051 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:33.848945 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" event={"ID":"a5071ae6-69e3-42fb-9b62-e35c5e89017f","Type":"ContainerDied","Data":"ced2a3f1b5c6a5348d05a402237c65d6cdb15c7e48133dd90378a42f242b38e6"} Jul 22 11:51:33.849277 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:33.849261 2534 scope.go:117] "RemoveContainer" containerID="ced2a3f1b5c6a5348d05a402237c65d6cdb15c7e48133dd90378a42f242b38e6" Jul 22 11:51:34.855210 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:34.855179 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nwgcx" event={"ID":"0ba8d0cb-2b77-48d8-b186-b4152f82d6f8","Type":"ContainerStarted","Data":"21718f458fd54905932c9f62330e4dab16a96ca0629170293d569c6151c687a2"} Jul 22 11:51:34.856780 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:34.856758 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nvc44" event={"ID":"a5071ae6-69e3-42fb-9b62-e35c5e89017f","Type":"ContainerStarted","Data":"94be768dcae10aa6c2820919655e4e7af5b4ae187b2905b4290d05ac4fccd5f7"} Jul 22 11:51:37.441063 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.441021 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" podUID="6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" containerName="registry" containerID="cri-o://872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f" gracePeriod=30 Jul 22 11:51:37.742046 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.741990 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:51:37.869711 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.869675 2534 generic.go:356] "Generic (PLEG): container finished" podID="6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" containerID="872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f" exitCode=0 Jul 22 11:51:37.869888 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.869751 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" event={"ID":"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a","Type":"ContainerDied","Data":"872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f"} Jul 22 11:51:37.869888 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.869774 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" Jul 22 11:51:37.869888 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.869841 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-578cbd896f-dfpt2" event={"ID":"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a","Type":"ContainerDied","Data":"e11808760f4d3e4161ed1328686228c32d41e69e32141e1aa5c0d1ffebfabc38"} Jul 22 11:51:37.869888 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.869879 2534 scope.go:117] "RemoveContainer" containerID="872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f" Jul 22 11:51:37.878180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.878155 2534 scope.go:117] "RemoveContainer" containerID="872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f" Jul 22 11:51:37.878497 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:51:37.878475 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f\": container with ID starting with 872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f not found: ID does not exist" containerID="872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f" Jul 22 11:51:37.878590 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.878509 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f"} err="failed to get container status \"872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f\": rpc error: code = NotFound desc = could not find container \"872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f\": container with ID starting with 872930456be033d6da02ac27d965b85bd9bb18d474000550dd8959028485632f not found: ID does not exist" Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.899998 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jffr4\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-kube-api-access-jffr4\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900065 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-image-registry-private-configuration\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900104 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-bound-sa-token\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900148 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-ca-trust-extracted\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900173 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900215 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-installation-pull-secrets\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900247 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-certificates\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900288 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-trusted-ca\") pod \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\" (UID: \"6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a\") " Jul 22 11:51:37.901115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.900829 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:51:37.906603 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.906550 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:51:37.906832 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.906778 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:51:37.908839 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.908764 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:51:37.909115 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.909089 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:51:37.909307 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.909283 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:51:37.909307 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.909283 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-kube-api-access-jffr4" (OuterVolumeSpecName: "kube-api-access-jffr4") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "kube-api-access-jffr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:51:37.911967 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:37.911941 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" (UID: "6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:51:38.001256 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001159 2534 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-image-registry-private-configuration\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.001475 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001455 2534 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-bound-sa-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.001583 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001570 2534 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-ca-trust-extracted\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.001673 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001660 2534 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-tls\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.001766 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001754 2534 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-installation-pull-secrets\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.001887 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001872 2534 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-registry-certificates\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.001979 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.001967 2534 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-trusted-ca\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.002062 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.002051 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jffr4\" (UniqueName: \"kubernetes.io/projected/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a-kube-api-access-jffr4\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:38.189774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.189735 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-578cbd896f-dfpt2"] Jul 22 11:51:38.196359 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:38.196332 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-578cbd896f-dfpt2"] Jul 22 11:51:40.100525 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:40.100488 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" path="/var/lib/kubelet/pods/6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a/volumes" Jul 22 11:51:46.053175 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.053135 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7484cf5f9b-h4jjw" podUID="e86cfe83-2a62-4af5-8e12-8a5276de2a21" containerName="console" containerID="cri-o://c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55" gracePeriod=15 Jul 22 11:51:46.318366 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.318305 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:51:46.469658 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.469628 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-oauth-serving-cert\") pod \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " Jul 22 11:51:46.469863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.469663 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-service-ca\") pod \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " Jul 22 11:51:46.469863 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.469726 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-oauth-config\") pod \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " Jul 22 11:51:46.469976 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.469896 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxknb\" (UniqueName: \"kubernetes.io/projected/e86cfe83-2a62-4af5-8e12-8a5276de2a21-kube-api-access-xxknb\") pod \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " Jul 22 11:51:46.469976 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.469959 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-serving-cert\") pod \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " Jul 22 11:51:46.470090 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.470005 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-config\") pod \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\" (UID: \"e86cfe83-2a62-4af5-8e12-8a5276de2a21\") " Jul 22 11:51:46.470142 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.470104 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-service-ca" (OuterVolumeSpecName: "service-ca") pod "e86cfe83-2a62-4af5-8e12-8a5276de2a21" (UID: "e86cfe83-2a62-4af5-8e12-8a5276de2a21"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:51:46.470142 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.470117 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e86cfe83-2a62-4af5-8e12-8a5276de2a21" (UID: "e86cfe83-2a62-4af5-8e12-8a5276de2a21"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:51:46.470277 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.470258 2534 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-oauth-serving-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:46.470357 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.470335 2534 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-service-ca\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:46.470675 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.470612 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-config" (OuterVolumeSpecName: "console-config") pod "e86cfe83-2a62-4af5-8e12-8a5276de2a21" (UID: "e86cfe83-2a62-4af5-8e12-8a5276de2a21"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:51:46.472171 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.472146 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e86cfe83-2a62-4af5-8e12-8a5276de2a21" (UID: "e86cfe83-2a62-4af5-8e12-8a5276de2a21"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:51:46.472258 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.472171 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86cfe83-2a62-4af5-8e12-8a5276de2a21-kube-api-access-xxknb" (OuterVolumeSpecName: "kube-api-access-xxknb") pod "e86cfe83-2a62-4af5-8e12-8a5276de2a21" (UID: "e86cfe83-2a62-4af5-8e12-8a5276de2a21"). InnerVolumeSpecName "kube-api-access-xxknb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:51:46.472258 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.472241 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e86cfe83-2a62-4af5-8e12-8a5276de2a21" (UID: "e86cfe83-2a62-4af5-8e12-8a5276de2a21"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:51:46.571591 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.571520 2534 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-serving-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:46.571591 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.571549 2534 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-config\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:46.571591 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.571560 2534 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86cfe83-2a62-4af5-8e12-8a5276de2a21-console-oauth-config\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:46.571591 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.571568 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxknb\" (UniqueName: \"kubernetes.io/projected/e86cfe83-2a62-4af5-8e12-8a5276de2a21-kube-api-access-xxknb\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:51:46.904699 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.904666 2534 generic.go:356] "Generic (PLEG): container finished" podID="e86cfe83-2a62-4af5-8e12-8a5276de2a21" containerID="c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55" exitCode=0 Jul 22 11:51:46.904881 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.904713 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7484cf5f9b-h4jjw" Jul 22 11:51:46.904881 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.904757 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7484cf5f9b-h4jjw" event={"ID":"e86cfe83-2a62-4af5-8e12-8a5276de2a21","Type":"ContainerDied","Data":"c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55"} Jul 22 11:51:46.904881 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.904824 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7484cf5f9b-h4jjw" event={"ID":"e86cfe83-2a62-4af5-8e12-8a5276de2a21","Type":"ContainerDied","Data":"4d9f84af26365d487fd93be00b4e11c23be12a876adde219f7e6aa14312897ca"} Jul 22 11:51:46.904881 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.904847 2534 scope.go:117] "RemoveContainer" containerID="c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55" Jul 22 11:51:46.914904 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.914889 2534 scope.go:117] "RemoveContainer" containerID="c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55" Jul 22 11:51:46.915163 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:51:46.915144 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55\": container with ID starting with c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55 not found: ID does not exist" containerID="c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55" Jul 22 11:51:46.915210 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.915170 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55"} err="failed to get container status \"c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55\": rpc error: code = NotFound desc = could not find container \"c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55\": container with ID starting with c4b848943a0024e891548043dc9cd340aa2e38cfddfe1cc0c9e2189723d62e55 not found: ID does not exist" Jul 22 11:51:46.928286 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.928260 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7484cf5f9b-h4jjw"] Jul 22 11:51:46.933152 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:46.933131 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7484cf5f9b-h4jjw"] Jul 22 11:51:48.100380 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:51:48.100334 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e86cfe83-2a62-4af5-8e12-8a5276de2a21" path="/var/lib/kubelet/pods/e86cfe83-2a62-4af5-8e12-8a5276de2a21/volumes" Jul 22 11:52:29.942626 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942546 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-644d6f7956-dmd4d"] Jul 22 11:52:29.943066 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942833 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" containerName="registry" Jul 22 11:52:29.943066 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942845 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" containerName="registry" Jul 22 11:52:29.943066 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942880 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86cfe83-2a62-4af5-8e12-8a5276de2a21" containerName="console" Jul 22 11:52:29.943066 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942886 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86cfe83-2a62-4af5-8e12-8a5276de2a21" containerName="console" Jul 22 11:52:29.943066 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942932 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d68bbc7-edbd-4e00-b07f-f6c33d7b5b5a" containerName="registry" Jul 22 11:52:29.943066 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.942940 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86cfe83-2a62-4af5-8e12-8a5276de2a21" containerName="console" Jul 22 11:52:29.946057 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.946036 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.949252 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.949230 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 22 11:52:29.949388 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.949294 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-57pc4\"" Jul 22 11:52:29.949436 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.949410 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 22 11:52:29.949557 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.949538 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 22 11:52:29.949641 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.949623 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 22 11:52:29.949763 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.949744 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 22 11:52:29.958302 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.957374 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 22 11:52:29.973685 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.973645 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-644d6f7956-dmd4d"] Jul 22 11:52:29.984613 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984588 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-secret-telemeter-client\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984710 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984625 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-telemeter-client-tls\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984710 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984651 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-telemeter-trusted-ca-bundle\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984710 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984689 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-metrics-client-ca\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984884 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984717 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-serving-certs-ca-bundle\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984884 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984734 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-federate-client-tls\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984884 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984752 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bsd4\" (UniqueName: \"kubernetes.io/projected/154a039d-250f-4a66-a6b2-9f7148424352-kube-api-access-8bsd4\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:29.984884 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:29.984776 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.085956 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.085924 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-secret-telemeter-client\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.085956 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.085963 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-telemeter-client-tls\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.085980 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-telemeter-trusted-ca-bundle\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.086001 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-metrics-client-ca\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.086033 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-serving-certs-ca-bundle\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.086060 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-federate-client-tls\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.086092 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bsd4\" (UniqueName: \"kubernetes.io/projected/154a039d-250f-4a66-a6b2-9f7148424352-kube-api-access-8bsd4\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.086137 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.086849 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.086823 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-metrics-client-ca\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.087351 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.087327 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-serving-certs-ca-bundle\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.087434 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.087372 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154a039d-250f-4a66-a6b2-9f7148424352-telemeter-trusted-ca-bundle\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.088749 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.088724 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-telemeter-client-tls\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.088859 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.088732 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-secret-telemeter-client\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.088859 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.088832 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.089597 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.089580 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/154a039d-250f-4a66-a6b2-9f7148424352-federate-client-tls\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.095689 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.095670 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bsd4\" (UniqueName: \"kubernetes.io/projected/154a039d-250f-4a66-a6b2-9f7148424352-kube-api-access-8bsd4\") pod \"telemeter-client-644d6f7956-dmd4d\" (UID: \"154a039d-250f-4a66-a6b2-9f7148424352\") " pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.261706 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.261623 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" Jul 22 11:52:30.402426 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:30.402382 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-644d6f7956-dmd4d"] Jul 22 11:52:30.405235 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:52:30.405206 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod154a039d_250f_4a66_a6b2_9f7148424352.slice/crio-9f7aa60f334bdd7f33e7881e8c19540c3dc026197c0268f2f8511a22845a4e0b WatchSource:0}: Error finding container 9f7aa60f334bdd7f33e7881e8c19540c3dc026197c0268f2f8511a22845a4e0b: Status 404 returned error can't find the container with id 9f7aa60f334bdd7f33e7881e8c19540c3dc026197c0268f2f8511a22845a4e0b Jul 22 11:52:31.064301 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:31.064269 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" event={"ID":"154a039d-250f-4a66-a6b2-9f7148424352","Type":"ContainerStarted","Data":"9f7aa60f334bdd7f33e7881e8c19540c3dc026197c0268f2f8511a22845a4e0b"} Jul 22 11:52:33.073573 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:33.073539 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" event={"ID":"154a039d-250f-4a66-a6b2-9f7148424352","Type":"ContainerStarted","Data":"0ed7095a80a07d7186dec5a65f9ef0d762a8a278dda887c87ce4d2bd54b59aa5"} Jul 22 11:52:34.079979 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.079888 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" event={"ID":"154a039d-250f-4a66-a6b2-9f7148424352","Type":"ContainerStarted","Data":"5e6d1e9ce79bb38105941f4f9f292668b9c58257d57a1c49c908c93009ea15db"} Jul 22 11:52:34.079979 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.079966 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" event={"ID":"154a039d-250f-4a66-a6b2-9f7148424352","Type":"ContainerStarted","Data":"09e2566e72bc255e6eff787dd6020f7d67fc647cb77ef900b89988f8482e26e2"} Jul 22 11:52:34.109293 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.109241 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-644d6f7956-dmd4d" podStartSLOduration=1.897678563 podStartE2EDuration="5.109225539s" totalImagesPullingTime="3.211546976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:52:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:52:34.106783795 +0000 UTC m=+182.432631073" watchObservedRunningTime="2026-07-22 11:52:34.109225539 +0000 UTC m=+182.435072803" Jul 22 11:52:34.823688 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.823648 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c6c84bd5d-xxn8k"] Jul 22 11:52:34.827425 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.827399 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.830478 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.830453 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sdcc8\"" Jul 22 11:52:34.830478 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.830453 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 11:52:34.830636 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.830591 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 11:52:34.831070 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.831054 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 11:52:34.831128 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.831071 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 11:52:34.831128 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.831074 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 11:52:34.836175 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.836157 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 11:52:34.840414 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.840395 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6c84bd5d-xxn8k"] Jul 22 11:52:34.928454 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928420 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-oauth-serving-cert\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.928454 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928454 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-config\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.928648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928486 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-serving-cert\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.928648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928569 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-oauth-config\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.928648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928608 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-trusted-ca-bundle\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.928648 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928626 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-service-ca\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:34.928820 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:34.928655 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkrfx\" (UniqueName: \"kubernetes.io/projected/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-kube-api-access-mkrfx\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029594 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029565 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-serving-cert\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029609 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-oauth-config\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029639 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-trusted-ca-bundle\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029663 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-service-ca\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029688 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkrfx\" (UniqueName: \"kubernetes.io/projected/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-kube-api-access-mkrfx\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029720 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-oauth-serving-cert\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.029771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.029749 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-config\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.030481 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.030457 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-config\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.030590 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.030534 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-oauth-serving-cert\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.030632 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.030590 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-service-ca\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.030678 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.030663 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-trusted-ca-bundle\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.032772 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.032752 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-serving-cert\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.032865 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.032770 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-oauth-config\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.043932 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.043912 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkrfx\" (UniqueName: \"kubernetes.io/projected/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-kube-api-access-mkrfx\") pod \"console-7c6c84bd5d-xxn8k\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.137726 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.137694 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:35.279596 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:35.279568 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6c84bd5d-xxn8k"] Jul 22 11:52:35.282268 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:52:35.282242 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23dab2b_71cb_4ccc_a518_5e5e2743b86d.slice/crio-6e10738a7a36f93bc90a89fbc1b52fdf0593f40122a1d0862c47930daf3621b1 WatchSource:0}: Error finding container 6e10738a7a36f93bc90a89fbc1b52fdf0593f40122a1d0862c47930daf3621b1: Status 404 returned error can't find the container with id 6e10738a7a36f93bc90a89fbc1b52fdf0593f40122a1d0862c47930daf3621b1 Jul 22 11:52:36.089284 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:36.089251 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c84bd5d-xxn8k" event={"ID":"a23dab2b-71cb-4ccc-a518-5e5e2743b86d","Type":"ContainerStarted","Data":"05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9"} Jul 22 11:52:36.089458 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:36.089300 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c84bd5d-xxn8k" event={"ID":"a23dab2b-71cb-4ccc-a518-5e5e2743b86d","Type":"ContainerStarted","Data":"6e10738a7a36f93bc90a89fbc1b52fdf0593f40122a1d0862c47930daf3621b1"} Jul 22 11:52:36.109869 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:36.109821 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c6c84bd5d-xxn8k" podStartSLOduration=2.109780436 podStartE2EDuration="2.109780436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:52:36.109484322 +0000 UTC m=+184.435331609" watchObservedRunningTime="2026-07-22 11:52:36.109780436 +0000 UTC m=+184.435627701" Jul 22 11:52:42.596386 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.596353 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6c84bd5d-xxn8k"] Jul 22 11:52:42.643281 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.643244 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-567d94866c-9c28x"] Jul 22 11:52:42.646650 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.646626 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.668148 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.668123 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567d94866c-9c28x"] Jul 22 11:52:42.693142 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693106 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-console-config\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.693318 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693149 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-oauth-serving-cert\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.693318 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693229 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-service-ca\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.693318 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693282 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-oauth-config\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.693479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693350 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-serving-cert\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.693479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693393 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-trusted-ca-bundle\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.693479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.693428 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbdxc\" (UniqueName: \"kubernetes.io/projected/e17dda2e-5c90-4922-b059-960d16a383b4-kube-api-access-dbdxc\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794130 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794088 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-serving-cert\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794130 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794129 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-trusted-ca-bundle\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794148 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbdxc\" (UniqueName: \"kubernetes.io/projected/e17dda2e-5c90-4922-b059-960d16a383b4-kube-api-access-dbdxc\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794185 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-console-config\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794201 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-oauth-serving-cert\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794218 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-service-ca\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.794362 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794250 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-oauth-config\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.795011 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.794981 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-console-config\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.795140 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.795049 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-trusted-ca-bundle\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.795140 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.795062 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-service-ca\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.795140 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.795100 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-oauth-serving-cert\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.796858 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.796827 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-oauth-config\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.796994 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.796959 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-serving-cert\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.805451 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.805426 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbdxc\" (UniqueName: \"kubernetes.io/projected/e17dda2e-5c90-4922-b059-960d16a383b4-kube-api-access-dbdxc\") pod \"console-567d94866c-9c28x\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:42.956771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:42.956734 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:43.092403 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:43.092377 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567d94866c-9c28x"] Jul 22 11:52:43.094782 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:52:43.094758 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode17dda2e_5c90_4922_b059_960d16a383b4.slice/crio-66cc45df40096415586d11ad9bc7f9c39c65a9a5694a529d600941a8894fb867 WatchSource:0}: Error finding container 66cc45df40096415586d11ad9bc7f9c39c65a9a5694a529d600941a8894fb867: Status 404 returned error can't find the container with id 66cc45df40096415586d11ad9bc7f9c39c65a9a5694a529d600941a8894fb867 Jul 22 11:52:43.118059 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:43.118031 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d94866c-9c28x" event={"ID":"e17dda2e-5c90-4922-b059-960d16a383b4","Type":"ContainerStarted","Data":"66cc45df40096415586d11ad9bc7f9c39c65a9a5694a529d600941a8894fb867"} Jul 22 11:52:44.123428 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:44.123392 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d94866c-9c28x" event={"ID":"e17dda2e-5c90-4922-b059-960d16a383b4","Type":"ContainerStarted","Data":"d941556da35888e0fbd6e51d6d298c2504f0c72fd15cc5aa7e3790b09fbf35e6"} Jul 22 11:52:44.145891 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:44.145840 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-567d94866c-9c28x" podStartSLOduration=2.145825768 podStartE2EDuration="2.145825768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:52:44.1448572 +0000 UTC m=+192.470704467" watchObservedRunningTime="2026-07-22 11:52:44.145825768 +0000 UTC m=+192.471673028" Jul 22 11:52:45.138481 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:45.138454 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:52:52.957080 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:52.957028 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:52.957485 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:52.957106 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:52.961954 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:52.961932 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:52:53.159807 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:52:53.159763 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:53:07.616836 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.616756 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c6c84bd5d-xxn8k" podUID="a23dab2b-71cb-4ccc-a518-5e5e2743b86d" containerName="console" containerID="cri-o://05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9" gracePeriod=15 Jul 22 11:53:07.865652 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.865623 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:53:07.904170 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904143 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-oauth-config\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:07.904360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904185 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-serving-cert\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:07.904360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904230 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkrfx\" (UniqueName: \"kubernetes.io/projected/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-kube-api-access-mkrfx\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:07.904360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904259 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-service-ca\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:07.904360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904282 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-oauth-serving-cert\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:07.904360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904316 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-trusted-ca-bundle\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:07.904715 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904678 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-service-ca" (OuterVolumeSpecName: "service-ca") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:53:07.904864 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.904748 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:53:07.905078 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.905058 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:53:07.906517 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.906497 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:53:07.906945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.906919 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-kube-api-access-mkrfx" (OuterVolumeSpecName: "kube-api-access-mkrfx") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "kube-api-access-mkrfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:53:07.906945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:07.906925 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:53:08.005322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005287 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-config\") pod \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\" (UID: \"a23dab2b-71cb-4ccc-a518-5e5e2743b86d\") " Jul 22 11:53:08.005518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005449 2534 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-serving-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.005518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005461 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkrfx\" (UniqueName: \"kubernetes.io/projected/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-kube-api-access-mkrfx\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.005518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005470 2534 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-service-ca\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.005518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005479 2534 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-oauth-serving-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.005518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005486 2534 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-trusted-ca-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.005518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005494 2534 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-oauth-config\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.005829 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.005705 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-config" (OuterVolumeSpecName: "console-config") pod "a23dab2b-71cb-4ccc-a518-5e5e2743b86d" (UID: "a23dab2b-71cb-4ccc-a518-5e5e2743b86d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:53:08.106649 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.106607 2534 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a23dab2b-71cb-4ccc-a518-5e5e2743b86d-console-config\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:08.216239 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.216148 2534 generic.go:356] "Generic (PLEG): container finished" podID="a23dab2b-71cb-4ccc-a518-5e5e2743b86d" containerID="05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9" exitCode=0 Jul 22 11:53:08.216239 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.216192 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c84bd5d-xxn8k" Jul 22 11:53:08.216427 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.216235 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c84bd5d-xxn8k" event={"ID":"a23dab2b-71cb-4ccc-a518-5e5e2743b86d","Type":"ContainerDied","Data":"05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9"} Jul 22 11:53:08.216427 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.216285 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c84bd5d-xxn8k" event={"ID":"a23dab2b-71cb-4ccc-a518-5e5e2743b86d","Type":"ContainerDied","Data":"6e10738a7a36f93bc90a89fbc1b52fdf0593f40122a1d0862c47930daf3621b1"} Jul 22 11:53:08.216427 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.216305 2534 scope.go:117] "RemoveContainer" containerID="05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9" Jul 22 11:53:08.224300 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.224284 2534 scope.go:117] "RemoveContainer" containerID="05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9" Jul 22 11:53:08.224543 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:53:08.224527 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9\": container with ID starting with 05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9 not found: ID does not exist" containerID="05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9" Jul 22 11:53:08.224605 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.224549 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9"} err="failed to get container status \"05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9\": rpc error: code = NotFound desc = could not find container \"05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9\": container with ID starting with 05383d1462963c6e46ea066e3406cb29c1af953f9accdc2cd617e324f9d105b9 not found: ID does not exist" Jul 22 11:53:08.235907 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.235885 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6c84bd5d-xxn8k"] Jul 22 11:53:08.243097 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:08.243077 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c6c84bd5d-xxn8k"] Jul 22 11:53:10.100860 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:10.100820 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a23dab2b-71cb-4ccc-a518-5e5e2743b86d" path="/var/lib/kubelet/pods/a23dab2b-71cb-4ccc-a518-5e5e2743b86d/volumes" Jul 22 11:53:25.694168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.694140 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz"] Jul 22 11:53:25.694559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.694440 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23dab2b-71cb-4ccc-a518-5e5e2743b86d" containerName="console" Jul 22 11:53:25.694559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.694452 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23dab2b-71cb-4ccc-a518-5e5e2743b86d" containerName="console" Jul 22 11:53:25.694559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.694509 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="a23dab2b-71cb-4ccc-a518-5e5e2743b86d" containerName="console" Jul 22 11:53:25.697474 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.697457 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.700137 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.700116 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 11:53:25.700232 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.700205 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d6mww\"" Jul 22 11:53:25.700655 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.700641 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 11:53:25.708944 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.708914 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz"] Jul 22 11:53:25.738773 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.738746 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.738942 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.738817 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.738942 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.738850 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqnft\" (UniqueName: \"kubernetes.io/projected/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-kube-api-access-sqnft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.839744 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.839704 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.839949 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.839754 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqnft\" (UniqueName: \"kubernetes.io/projected/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-kube-api-access-sqnft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.839949 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.839834 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.840184 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.840161 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.840262 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.840166 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:25.850497 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:25.850471 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqnft\" (UniqueName: \"kubernetes.io/projected/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-kube-api-access-sqnft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:26.006616 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:26.006530 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:26.146739 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:53:26.146715 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ad6514_ad9d_4c42_abdc_87c7e8a85a4a.slice/crio-31e3af4ff4ddb2fa14d331dcd90791030ad12149b159f79e9317523fc8780e58 WatchSource:0}: Error finding container 31e3af4ff4ddb2fa14d331dcd90791030ad12149b159f79e9317523fc8780e58: Status 404 returned error can't find the container with id 31e3af4ff4ddb2fa14d331dcd90791030ad12149b159f79e9317523fc8780e58 Jul 22 11:53:26.147939 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:26.147913 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz"] Jul 22 11:53:26.286941 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:26.286849 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" event={"ID":"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a","Type":"ContainerStarted","Data":"31e3af4ff4ddb2fa14d331dcd90791030ad12149b159f79e9317523fc8780e58"} Jul 22 11:53:33.317635 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:33.317599 2534 generic.go:356] "Generic (PLEG): container finished" podID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerID="7ef7b73399b111168c341803bcc027a595e75486d7ce579cf0f9a295151a7731" exitCode=0 Jul 22 11:53:33.318056 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:33.317680 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" event={"ID":"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a","Type":"ContainerDied","Data":"7ef7b73399b111168c341803bcc027a595e75486d7ce579cf0f9a295151a7731"} Jul 22 11:53:36.333319 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:36.333284 2534 generic.go:356] "Generic (PLEG): container finished" podID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerID="9efb1b22f74ac780733786f98a83600afe48e908623de011aceccfb6c7219b53" exitCode=0 Jul 22 11:53:36.333725 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:36.333326 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" event={"ID":"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a","Type":"ContainerDied","Data":"9efb1b22f74ac780733786f98a83600afe48e908623de011aceccfb6c7219b53"} Jul 22 11:53:43.364810 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:43.364766 2534 generic.go:356] "Generic (PLEG): container finished" podID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerID="c076f77573061e452d85ba512855f8deda24a432080ac59d9d60a84b5c9b1772" exitCode=0 Jul 22 11:53:43.365219 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:43.364820 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" event={"ID":"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a","Type":"ContainerDied","Data":"c076f77573061e452d85ba512855f8deda24a432080ac59d9d60a84b5c9b1772"} Jul 22 11:53:44.498238 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.498217 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:44.601624 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.601595 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-util\") pod \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " Jul 22 11:53:44.601624 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.601630 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqnft\" (UniqueName: \"kubernetes.io/projected/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-kube-api-access-sqnft\") pod \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " Jul 22 11:53:44.601875 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.601656 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-bundle\") pod \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\" (UID: \"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a\") " Jul 22 11:53:44.602449 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.602338 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-bundle" (OuterVolumeSpecName: "bundle") pod "e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" (UID: "e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:44.604020 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.603993 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-kube-api-access-sqnft" (OuterVolumeSpecName: "kube-api-access-sqnft") pod "e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" (UID: "e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a"). InnerVolumeSpecName "kube-api-access-sqnft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:53:44.606985 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.606960 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-util" (OuterVolumeSpecName: "util") pod "e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" (UID: "e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:44.703132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.703106 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:44.703132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.703129 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqnft\" (UniqueName: \"kubernetes.io/projected/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-kube-api-access-sqnft\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:44.703132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:44.703138 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:53:45.374432 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:45.374406 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" Jul 22 11:53:45.374599 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:45.374406 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407xttz" event={"ID":"e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a","Type":"ContainerDied","Data":"31e3af4ff4ddb2fa14d331dcd90791030ad12149b159f79e9317523fc8780e58"} Jul 22 11:53:45.374599 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:45.374519 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31e3af4ff4ddb2fa14d331dcd90791030ad12149b159f79e9317523fc8780e58" Jul 22 11:53:49.511917 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.511885 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf"] Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512169 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="pull" Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512180 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="pull" Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512190 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="extract" Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512195 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="extract" Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512207 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="util" Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512213 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="util" Jul 22 11:53:49.512308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.512260 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ad6514-ad9d-4c42-abdc-87c7e8a85a4a" containerName="extract" Jul 22 11:53:49.567703 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.567677 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.571707 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.571687 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8jhz8\"" Jul 22 11:53:49.571859 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.571822 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf"] Jul 22 11:53:49.572645 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.572627 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:53:49.574377 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.574355 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 11:53:49.644010 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.643978 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2js78\" (UniqueName: \"kubernetes.io/projected/87a3a2b6-4e1c-462d-9b66-020fd282f4b8-kube-api-access-2js78\") pod \"cert-manager-operator-controller-manager-55c444bf86-62tvf\" (UID: \"87a3a2b6-4e1c-462d-9b66-020fd282f4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.644162 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.644019 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87a3a2b6-4e1c-462d-9b66-020fd282f4b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-62tvf\" (UID: \"87a3a2b6-4e1c-462d-9b66-020fd282f4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.744820 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.744751 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2js78\" (UniqueName: \"kubernetes.io/projected/87a3a2b6-4e1c-462d-9b66-020fd282f4b8-kube-api-access-2js78\") pod \"cert-manager-operator-controller-manager-55c444bf86-62tvf\" (UID: \"87a3a2b6-4e1c-462d-9b66-020fd282f4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.745031 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.744846 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87a3a2b6-4e1c-462d-9b66-020fd282f4b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-62tvf\" (UID: \"87a3a2b6-4e1c-462d-9b66-020fd282f4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.745246 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.745227 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87a3a2b6-4e1c-462d-9b66-020fd282f4b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-62tvf\" (UID: \"87a3a2b6-4e1c-462d-9b66-020fd282f4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.759218 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.759180 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2js78\" (UniqueName: \"kubernetes.io/projected/87a3a2b6-4e1c-462d-9b66-020fd282f4b8-kube-api-access-2js78\") pod \"cert-manager-operator-controller-manager-55c444bf86-62tvf\" (UID: \"87a3a2b6-4e1c-462d-9b66-020fd282f4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:49.878144 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:49.878111 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" Jul 22 11:53:50.155145 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:50.155057 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf"] Jul 22 11:53:50.165667 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:53:50.165640 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a3a2b6_4e1c_462d_9b66_020fd282f4b8.slice/crio-b2a965e9d1a75ba9670c25b57275a35ef0d7d2d9372a8f0c1eae8994cab96f4f WatchSource:0}: Error finding container b2a965e9d1a75ba9670c25b57275a35ef0d7d2d9372a8f0c1eae8994cab96f4f: Status 404 returned error can't find the container with id b2a965e9d1a75ba9670c25b57275a35ef0d7d2d9372a8f0c1eae8994cab96f4f Jul 22 11:53:50.396588 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:50.396556 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" event={"ID":"87a3a2b6-4e1c-462d-9b66-020fd282f4b8","Type":"ContainerStarted","Data":"b2a965e9d1a75ba9670c25b57275a35ef0d7d2d9372a8f0c1eae8994cab96f4f"} Jul 22 11:53:52.408317 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:52.408277 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" event={"ID":"87a3a2b6-4e1c-462d-9b66-020fd282f4b8","Type":"ContainerStarted","Data":"3ee37e4f12499fbd25ce39a6082a57dc5d265ac38b042e629e44acbcddfcc3a4"} Jul 22 11:53:52.460828 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:52.460749 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62tvf" podStartSLOduration=1.352913901 podStartE2EDuration="3.460728721s" totalImagesPullingTime="2.10781482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:53:52.457139455 +0000 UTC m=+260.782986719" watchObservedRunningTime="2026-07-22 11:53:52.460728721 +0000 UTC m=+260.786575986" Jul 22 11:53:54.087606 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.087521 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628"] Jul 22 11:53:54.113272 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.113238 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.121133 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.121108 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 11:53:54.121686 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.121550 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628"] Jul 22 11:53:54.121759 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.121712 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 11:53:54.130404 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.130381 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d6mww\"" Jul 22 11:53:54.185197 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.185163 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.185197 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.185204 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.185417 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.185221 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwxml\" (UniqueName: \"kubernetes.io/projected/7a959c7f-63a5-4d46-93ac-0a26d638d800-kube-api-access-vwxml\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.286435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.286403 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.286435 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.286436 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.286626 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.286456 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwxml\" (UniqueName: \"kubernetes.io/projected/7a959c7f-63a5-4d46-93ac-0a26d638d800-kube-api-access-vwxml\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.286815 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.286775 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.286896 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.286870 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.297738 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.297703 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwxml\" (UniqueName: \"kubernetes.io/projected/7a959c7f-63a5-4d46-93ac-0a26d638d800-kube-api-access-vwxml\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.424992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.424951 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:53:54.595481 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:54.595426 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628"] Jul 22 11:53:54.598366 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:53:54.598336 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a959c7f_63a5_4d46_93ac_0a26d638d800.slice/crio-141656bb6ec73ac4a96a4fe9da616eb285802d085eaf34716fc688e82df34cb3 WatchSource:0}: Error finding container 141656bb6ec73ac4a96a4fe9da616eb285802d085eaf34716fc688e82df34cb3: Status 404 returned error can't find the container with id 141656bb6ec73ac4a96a4fe9da616eb285802d085eaf34716fc688e82df34cb3 Jul 22 11:53:55.424162 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:55.424128 2534 generic.go:356] "Generic (PLEG): container finished" podID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerID="73063dd99ffe6d9feb733763a700b642fc90ecb74a1259cd620d0ccac29bc4fa" exitCode=0 Jul 22 11:53:55.424542 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:55.424211 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" event={"ID":"7a959c7f-63a5-4d46-93ac-0a26d638d800","Type":"ContainerDied","Data":"73063dd99ffe6d9feb733763a700b642fc90ecb74a1259cd620d0ccac29bc4fa"} Jul 22 11:53:55.424542 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:55.424256 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" event={"ID":"7a959c7f-63a5-4d46-93ac-0a26d638d800","Type":"ContainerStarted","Data":"141656bb6ec73ac4a96a4fe9da616eb285802d085eaf34716fc688e82df34cb3"} Jul 22 11:53:56.687615 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.687581 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7"] Jul 22 11:53:56.690811 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.690767 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:56.695288 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.695267 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 11:53:56.695410 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.695270 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 11:53:56.695410 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.695270 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8mtch\"" Jul 22 11:53:56.733497 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.733461 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7"] Jul 22 11:53:56.805691 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.805652 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v988k\" (UniqueName: \"kubernetes.io/projected/66d61f25-5fe5-421c-8b0a-80195e842650-kube-api-access-v988k\") pod \"cert-manager-cainjector-559cdd8c87-jzgw7\" (UID: \"66d61f25-5fe5-421c-8b0a-80195e842650\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:56.805884 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.805745 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66d61f25-5fe5-421c-8b0a-80195e842650-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jzgw7\" (UID: \"66d61f25-5fe5-421c-8b0a-80195e842650\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:56.907269 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.907231 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v988k\" (UniqueName: \"kubernetes.io/projected/66d61f25-5fe5-421c-8b0a-80195e842650-kube-api-access-v988k\") pod \"cert-manager-cainjector-559cdd8c87-jzgw7\" (UID: \"66d61f25-5fe5-421c-8b0a-80195e842650\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:56.907442 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.907312 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66d61f25-5fe5-421c-8b0a-80195e842650-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jzgw7\" (UID: \"66d61f25-5fe5-421c-8b0a-80195e842650\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:56.931865 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.931840 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66d61f25-5fe5-421c-8b0a-80195e842650-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jzgw7\" (UID: \"66d61f25-5fe5-421c-8b0a-80195e842650\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:56.935771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:56.935744 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v988k\" (UniqueName: \"kubernetes.io/projected/66d61f25-5fe5-421c-8b0a-80195e842650-kube-api-access-v988k\") pod \"cert-manager-cainjector-559cdd8c87-jzgw7\" (UID: \"66d61f25-5fe5-421c-8b0a-80195e842650\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:57.006616 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:57.006520 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" Jul 22 11:53:57.534854 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:57.534823 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7"] Jul 22 11:53:57.538926 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:53:57.538901 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66d61f25_5fe5_421c_8b0a_80195e842650.slice/crio-da0e1d1681f234dbc65b733540d026b6b8217d01b997846a8aa42bfb798ac8fd WatchSource:0}: Error finding container da0e1d1681f234dbc65b733540d026b6b8217d01b997846a8aa42bfb798ac8fd: Status 404 returned error can't find the container with id da0e1d1681f234dbc65b733540d026b6b8217d01b997846a8aa42bfb798ac8fd Jul 22 11:53:58.441180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:58.441141 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" event={"ID":"66d61f25-5fe5-421c-8b0a-80195e842650","Type":"ContainerStarted","Data":"da0e1d1681f234dbc65b733540d026b6b8217d01b997846a8aa42bfb798ac8fd"} Jul 22 11:53:58.443450 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:58.443424 2534 generic.go:356] "Generic (PLEG): container finished" podID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerID="3a5755ae036b876daa2ddbaa05666ea5fd9ea9f6a8726ab4d83a060b9b8ab26c" exitCode=0 Jul 22 11:53:58.443567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:58.443508 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" event={"ID":"7a959c7f-63a5-4d46-93ac-0a26d638d800","Type":"ContainerDied","Data":"3a5755ae036b876daa2ddbaa05666ea5fd9ea9f6a8726ab4d83a060b9b8ab26c"} Jul 22 11:53:59.454327 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:59.454290 2534 generic.go:356] "Generic (PLEG): container finished" podID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerID="b172c1e018a3d72fbc3b20e54a74cf6272b9565fc39b4611c307680839a2dcbe" exitCode=0 Jul 22 11:53:59.454830 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:53:59.454416 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" event={"ID":"7a959c7f-63a5-4d46-93ac-0a26d638d800","Type":"ContainerDied","Data":"b172c1e018a3d72fbc3b20e54a74cf6272b9565fc39b4611c307680839a2dcbe"} Jul 22 11:54:00.461719 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.461683 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" event={"ID":"66d61f25-5fe5-421c-8b0a-80195e842650","Type":"ContainerStarted","Data":"a074958c53991170d107f6bf04e37a46bfb5393fc402486f29afb3b5d70d9871"} Jul 22 11:54:00.486871 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.486269 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jzgw7" podStartSLOduration=2.036472309 podStartE2EDuration="4.486245486s" totalImagesPullingTime="2.449773177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:54:00.485023701 +0000 UTC m=+268.810870965" watchObservedRunningTime="2026-07-22 11:54:00.486245486 +0000 UTC m=+268.812092750" Jul 22 11:54:00.599050 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.599024 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:54:00.741244 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.741154 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwxml\" (UniqueName: \"kubernetes.io/projected/7a959c7f-63a5-4d46-93ac-0a26d638d800-kube-api-access-vwxml\") pod \"7a959c7f-63a5-4d46-93ac-0a26d638d800\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " Jul 22 11:54:00.741244 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.741212 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-bundle\") pod \"7a959c7f-63a5-4d46-93ac-0a26d638d800\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " Jul 22 11:54:00.741244 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.741247 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-util\") pod \"7a959c7f-63a5-4d46-93ac-0a26d638d800\" (UID: \"7a959c7f-63a5-4d46-93ac-0a26d638d800\") " Jul 22 11:54:00.741657 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.741635 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-bundle" (OuterVolumeSpecName: "bundle") pod "7a959c7f-63a5-4d46-93ac-0a26d638d800" (UID: "7a959c7f-63a5-4d46-93ac-0a26d638d800"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:54:00.743497 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.743472 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a959c7f-63a5-4d46-93ac-0a26d638d800-kube-api-access-vwxml" (OuterVolumeSpecName: "kube-api-access-vwxml") pod "7a959c7f-63a5-4d46-93ac-0a26d638d800" (UID: "7a959c7f-63a5-4d46-93ac-0a26d638d800"). InnerVolumeSpecName "kube-api-access-vwxml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:54:00.745248 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.745227 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-util" (OuterVolumeSpecName: "util") pod "7a959c7f-63a5-4d46-93ac-0a26d638d800" (UID: "7a959c7f-63a5-4d46-93ac-0a26d638d800"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:54:00.842091 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.842051 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:54:00.842091 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.842083 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a959c7f-63a5-4d46-93ac-0a26d638d800-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:54:00.842091 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:00.842093 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwxml\" (UniqueName: \"kubernetes.io/projected/7a959c7f-63a5-4d46-93ac-0a26d638d800-kube-api-access-vwxml\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:54:01.468312 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:01.468283 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" Jul 22 11:54:01.468312 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:01.468294 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9g628" event={"ID":"7a959c7f-63a5-4d46-93ac-0a26d638d800","Type":"ContainerDied","Data":"141656bb6ec73ac4a96a4fe9da616eb285802d085eaf34716fc688e82df34cb3"} Jul 22 11:54:01.468763 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:01.468341 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="141656bb6ec73ac4a96a4fe9da616eb285802d085eaf34716fc688e82df34cb3" Jul 22 11:54:06.270078 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270037 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln"] Jul 22 11:54:06.270513 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270496 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="pull" Jul 22 11:54:06.270559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270518 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="pull" Jul 22 11:54:06.270559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270533 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="extract" Jul 22 11:54:06.270559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270543 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="extract" Jul 22 11:54:06.270680 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270578 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="util" Jul 22 11:54:06.270680 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270587 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="util" Jul 22 11:54:06.270680 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.270663 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a959c7f-63a5-4d46-93ac-0a26d638d800" containerName="extract" Jul 22 11:54:06.274556 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.274536 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.277638 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.277616 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:54:06.278575 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.278557 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 11:54:06.278682 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.278573 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rb8ft\"" Jul 22 11:54:06.293380 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.293344 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln"] Jul 22 11:54:06.392411 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.392377 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8efb5175-8aa4-42b1-b041-106d38e61751-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wzsln\" (UID: \"8efb5175-8aa4-42b1-b041-106d38e61751\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.392411 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.392413 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sr5t\" (UniqueName: \"kubernetes.io/projected/8efb5175-8aa4-42b1-b041-106d38e61751-kube-api-access-8sr5t\") pod \"openshift-lws-operator-fd8ccff4c-wzsln\" (UID: \"8efb5175-8aa4-42b1-b041-106d38e61751\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.493708 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.493679 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8efb5175-8aa4-42b1-b041-106d38e61751-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wzsln\" (UID: \"8efb5175-8aa4-42b1-b041-106d38e61751\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.493708 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.493714 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sr5t\" (UniqueName: \"kubernetes.io/projected/8efb5175-8aa4-42b1-b041-106d38e61751-kube-api-access-8sr5t\") pod \"openshift-lws-operator-fd8ccff4c-wzsln\" (UID: \"8efb5175-8aa4-42b1-b041-106d38e61751\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.494096 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.494075 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8efb5175-8aa4-42b1-b041-106d38e61751-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wzsln\" (UID: \"8efb5175-8aa4-42b1-b041-106d38e61751\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.506418 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.506399 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sr5t\" (UniqueName: \"kubernetes.io/projected/8efb5175-8aa4-42b1-b041-106d38e61751-kube-api-access-8sr5t\") pod \"openshift-lws-operator-fd8ccff4c-wzsln\" (UID: \"8efb5175-8aa4-42b1-b041-106d38e61751\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.586383 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.586304 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" Jul 22 11:54:06.729689 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:06.729662 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln"] Jul 22 11:54:06.731624 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:54:06.731588 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8efb5175_8aa4_42b1_b041_106d38e61751.slice/crio-cb7c59ee635d3c3ac2d5acc7a02da51c062d81ea0c93ee542908a987a498ba1c WatchSource:0}: Error finding container cb7c59ee635d3c3ac2d5acc7a02da51c062d81ea0c93ee542908a987a498ba1c: Status 404 returned error can't find the container with id cb7c59ee635d3c3ac2d5acc7a02da51c062d81ea0c93ee542908a987a498ba1c Jul 22 11:54:07.500064 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:07.500010 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" event={"ID":"8efb5175-8aa4-42b1-b041-106d38e61751","Type":"ContainerStarted","Data":"cb7c59ee635d3c3ac2d5acc7a02da51c062d81ea0c93ee542908a987a498ba1c"} Jul 22 11:54:09.511875 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:09.511839 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" event={"ID":"8efb5175-8aa4-42b1-b041-106d38e61751","Type":"ContainerStarted","Data":"fbb5944d23fe17f849cf00da3b12fbd179857c8355a524be8e23a12c9337831e"} Jul 22 11:54:09.535917 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:09.535867 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wzsln" podStartSLOduration=1.777075559 podStartE2EDuration="3.535849769s" totalImagesPullingTime="1.75877421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:54:09.534873949 +0000 UTC m=+277.860721214" watchObservedRunningTime="2026-07-22 11:54:09.535849769 +0000 UTC m=+277.861697033" Jul 22 11:54:12.238398 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.238363 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5"] Jul 22 11:54:12.241882 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.241866 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.245555 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.245535 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d6mww\"" Jul 22 11:54:12.245784 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.245771 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 11:54:12.246371 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.246358 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 11:54:12.255831 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.255808 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5"] Jul 22 11:54:12.338595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.338558 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.338750 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.338621 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.338750 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.338663 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ggt5\" (UniqueName: \"kubernetes.io/projected/c6d60647-b66b-4bb6-aa48-e581140ed445-kube-api-access-5ggt5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.440120 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.440035 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.440120 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.440085 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ggt5\" (UniqueName: \"kubernetes.io/projected/c6d60647-b66b-4bb6-aa48-e581140ed445-kube-api-access-5ggt5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.440300 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.440127 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.440456 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.440438 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.440500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.440471 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.450698 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.450669 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ggt5\" (UniqueName: \"kubernetes.io/projected/c6d60647-b66b-4bb6-aa48-e581140ed445-kube-api-access-5ggt5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.550930 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.550900 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:12.709173 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:12.707654 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5"] Jul 22 11:54:12.713144 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:54:12.713112 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d60647_b66b_4bb6_aa48_e581140ed445.slice/crio-8ec3f7ec5eb7fce265b4b8f3ca128d0c8a2a6f778cfcc007c64f29ccd8910449 WatchSource:0}: Error finding container 8ec3f7ec5eb7fce265b4b8f3ca128d0c8a2a6f778cfcc007c64f29ccd8910449: Status 404 returned error can't find the container with id 8ec3f7ec5eb7fce265b4b8f3ca128d0c8a2a6f778cfcc007c64f29ccd8910449 Jul 22 11:54:13.530467 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:13.530436 2534 generic.go:356] "Generic (PLEG): container finished" podID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerID="2dd48a931f3bbdbb4299dc103952db09494463037ac6ad229ab7b5bf33d041bd" exitCode=0 Jul 22 11:54:13.530885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:13.530517 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" event={"ID":"c6d60647-b66b-4bb6-aa48-e581140ed445","Type":"ContainerDied","Data":"2dd48a931f3bbdbb4299dc103952db09494463037ac6ad229ab7b5bf33d041bd"} Jul 22 11:54:13.530885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:13.530568 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" event={"ID":"c6d60647-b66b-4bb6-aa48-e581140ed445","Type":"ContainerStarted","Data":"8ec3f7ec5eb7fce265b4b8f3ca128d0c8a2a6f778cfcc007c64f29ccd8910449"} Jul 22 11:54:14.536518 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:14.536433 2534 generic.go:356] "Generic (PLEG): container finished" podID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerID="1f9e389aaa57484957dcd0019d4049a420192e4f26703215bbeca38bde982931" exitCode=0 Jul 22 11:54:14.536892 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:14.536517 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" event={"ID":"c6d60647-b66b-4bb6-aa48-e581140ed445","Type":"ContainerDied","Data":"1f9e389aaa57484957dcd0019d4049a420192e4f26703215bbeca38bde982931"} Jul 22 11:54:15.545397 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:15.545361 2534 generic.go:356] "Generic (PLEG): container finished" podID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerID="ad6e36dc28a97ee59fab12f1993ae371677d5a4b63b1b90ce92499acdebd87db" exitCode=0 Jul 22 11:54:15.545829 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:15.545448 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" event={"ID":"c6d60647-b66b-4bb6-aa48-e581140ed445","Type":"ContainerDied","Data":"ad6e36dc28a97ee59fab12f1993ae371677d5a4b63b1b90ce92499acdebd87db"} Jul 22 11:54:16.685437 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.685411 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:16.777027 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.776992 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-bundle\") pod \"c6d60647-b66b-4bb6-aa48-e581140ed445\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " Jul 22 11:54:16.777192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.777051 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ggt5\" (UniqueName: \"kubernetes.io/projected/c6d60647-b66b-4bb6-aa48-e581140ed445-kube-api-access-5ggt5\") pod \"c6d60647-b66b-4bb6-aa48-e581140ed445\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " Jul 22 11:54:16.777192 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.777070 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-util\") pod \"c6d60647-b66b-4bb6-aa48-e581140ed445\" (UID: \"c6d60647-b66b-4bb6-aa48-e581140ed445\") " Jul 22 11:54:16.777854 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.777829 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-bundle" (OuterVolumeSpecName: "bundle") pod "c6d60647-b66b-4bb6-aa48-e581140ed445" (UID: "c6d60647-b66b-4bb6-aa48-e581140ed445"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:54:16.779344 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.779323 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d60647-b66b-4bb6-aa48-e581140ed445-kube-api-access-5ggt5" (OuterVolumeSpecName: "kube-api-access-5ggt5") pod "c6d60647-b66b-4bb6-aa48-e581140ed445" (UID: "c6d60647-b66b-4bb6-aa48-e581140ed445"). InnerVolumeSpecName "kube-api-access-5ggt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:54:16.782583 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.782559 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-util" (OuterVolumeSpecName: "util") pod "c6d60647-b66b-4bb6-aa48-e581140ed445" (UID: "c6d60647-b66b-4bb6-aa48-e581140ed445"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:54:16.878309 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.878280 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:54:16.878309 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.878309 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ggt5\" (UniqueName: \"kubernetes.io/projected/c6d60647-b66b-4bb6-aa48-e581140ed445-kube-api-access-5ggt5\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:54:16.878488 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:16.878319 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6d60647-b66b-4bb6-aa48-e581140ed445-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:54:17.561111 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:17.561085 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" Jul 22 11:54:17.561111 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:17.561093 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbldj5" event={"ID":"c6d60647-b66b-4bb6-aa48-e581140ed445","Type":"ContainerDied","Data":"8ec3f7ec5eb7fce265b4b8f3ca128d0c8a2a6f778cfcc007c64f29ccd8910449"} Jul 22 11:54:17.561308 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:17.561136 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ec3f7ec5eb7fce265b4b8f3ca128d0c8a2a6f778cfcc007c64f29ccd8910449" Jul 22 11:54:25.250229 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250197 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c"] Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250479 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="util" Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250489 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="util" Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250496 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="extract" Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250502 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="extract" Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250526 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="pull" Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250535 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="pull" Jul 22 11:54:25.250764 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.250588 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d60647-b66b-4bb6-aa48-e581140ed445" containerName="extract" Jul 22 11:54:25.253786 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.253771 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.263234 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.263211 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 11:54:25.263367 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.263275 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 11:54:25.263367 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.263216 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 11:54:25.263367 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.263319 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 11:54:25.264038 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.264023 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x6j7c\"" Jul 22 11:54:25.295040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.295012 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c"] Jul 22 11:54:25.351809 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.351761 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/779ecd9c-4776-4fd1-becc-dbf7127314b4-tmp\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.352013 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.351838 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/779ecd9c-4776-4fd1-becc-dbf7127314b4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.352013 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.351913 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779ecd9c-4776-4fd1-becc-dbf7127314b4-webhook-cert\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.352013 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.351969 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/779ecd9c-4776-4fd1-becc-dbf7127314b4-manifests\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.352013 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.351996 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw2lk\" (UniqueName: \"kubernetes.io/projected/779ecd9c-4776-4fd1-becc-dbf7127314b4-kube-api-access-gw2lk\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.452834 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.452778 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779ecd9c-4776-4fd1-becc-dbf7127314b4-webhook-cert\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.453026 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.452848 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/779ecd9c-4776-4fd1-becc-dbf7127314b4-manifests\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.453026 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.452870 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw2lk\" (UniqueName: \"kubernetes.io/projected/779ecd9c-4776-4fd1-becc-dbf7127314b4-kube-api-access-gw2lk\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.453026 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.452897 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/779ecd9c-4776-4fd1-becc-dbf7127314b4-tmp\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.453026 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.452959 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/779ecd9c-4776-4fd1-becc-dbf7127314b4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.453377 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.453352 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/779ecd9c-4776-4fd1-becc-dbf7127314b4-manifests\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.455255 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.455228 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/779ecd9c-4776-4fd1-becc-dbf7127314b4-tmp\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.455597 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.455576 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/779ecd9c-4776-4fd1-becc-dbf7127314b4-webhook-cert\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.455680 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.455579 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/779ecd9c-4776-4fd1-becc-dbf7127314b4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.466222 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.466201 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw2lk\" (UniqueName: \"kubernetes.io/projected/779ecd9c-4776-4fd1-becc-dbf7127314b4-kube-api-access-gw2lk\") pod \"opendatahub-operator-controller-manager-c4d6d56dd-cvj4c\" (UID: \"779ecd9c-4776-4fd1-becc-dbf7127314b4\") " pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.563604 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.563517 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:25.725947 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:25.725924 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c"] Jul 22 11:54:25.727778 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:54:25.727753 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779ecd9c_4776_4fd1_becc_dbf7127314b4.slice/crio-ac05ae2018507d8b2d44d5cbbf8ac13283bbaf2441ba31e145c0196e1d4055f7 WatchSource:0}: Error finding container ac05ae2018507d8b2d44d5cbbf8ac13283bbaf2441ba31e145c0196e1d4055f7: Status 404 returned error can't find the container with id ac05ae2018507d8b2d44d5cbbf8ac13283bbaf2441ba31e145c0196e1d4055f7 Jul 22 11:54:26.607818 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:26.607720 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" event={"ID":"779ecd9c-4776-4fd1-becc-dbf7127314b4","Type":"ContainerStarted","Data":"ac05ae2018507d8b2d44d5cbbf8ac13283bbaf2441ba31e145c0196e1d4055f7"} Jul 22 11:54:29.628177 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:29.628138 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" event={"ID":"779ecd9c-4776-4fd1-becc-dbf7127314b4","Type":"ContainerStarted","Data":"608b375dd41c9688d0dd9cd91fac14f6adb57a7a33ac9a17f08813177f0247c6"} Jul 22 11:54:30.634436 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:30.634403 2534 generic.go:356] "Generic (PLEG): container finished" podID="779ecd9c-4776-4fd1-becc-dbf7127314b4" containerID="608b375dd41c9688d0dd9cd91fac14f6adb57a7a33ac9a17f08813177f0247c6" exitCode=0 Jul 22 11:54:30.634844 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:30.634500 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" event={"ID":"779ecd9c-4776-4fd1-becc-dbf7127314b4","Type":"ContainerDied","Data":"608b375dd41c9688d0dd9cd91fac14f6adb57a7a33ac9a17f08813177f0247c6"} Jul 22 11:54:31.641040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:31.641008 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" event={"ID":"779ecd9c-4776-4fd1-becc-dbf7127314b4","Type":"ContainerStarted","Data":"64b87fa3edc10ee79e74149fbb6f98561341c67347da27a9cc3c1920b3412c63"} Jul 22 11:54:31.641420 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:31.641115 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:31.668340 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:31.668290 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" podStartSLOduration=2.235468093 podStartE2EDuration="6.66827326s" totalImagesPullingTime="3.375268452s" totalInitContainerRuntime="1.057536715s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:54:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:54:31.666698337 +0000 UTC m=+299.992545600" watchObservedRunningTime="2026-07-22 11:54:31.66827326 +0000 UTC m=+299.994120524" Jul 22 11:54:32.011875 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:32.011783 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:54:32.023925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:32.023892 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:54:32.030483 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:32.030458 2534 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 11:54:42.648119 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:42.648090 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c4d6d56dd-cvj4c" Jul 22 11:54:43.856040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.856008 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-755b658494-fj45k"] Jul 22 11:54:43.860305 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.860276 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:43.868623 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.868602 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hz6hd\"" Jul 22 11:54:43.868735 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.868642 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 11:54:43.869107 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.869090 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 11:54:43.869389 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.869375 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 11:54:43.885469 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:43.885440 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-755b658494-fj45k"] Jul 22 11:54:44.012948 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.012905 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx48n\" (UniqueName: \"kubernetes.io/projected/f58d8345-352f-4c5e-b400-56dbc0546501-kube-api-access-dx48n\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.013112 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.012958 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f58d8345-352f-4c5e-b400-56dbc0546501-manager-config\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.013112 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.013033 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f58d8345-352f-4c5e-b400-56dbc0546501-metrics-cert\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.013112 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.013054 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f58d8345-352f-4c5e-b400-56dbc0546501-cert\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.113949 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.113867 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx48n\" (UniqueName: \"kubernetes.io/projected/f58d8345-352f-4c5e-b400-56dbc0546501-kube-api-access-dx48n\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.113949 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.113915 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f58d8345-352f-4c5e-b400-56dbc0546501-manager-config\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.114191 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.114172 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f58d8345-352f-4c5e-b400-56dbc0546501-metrics-cert\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.114230 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.114211 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f58d8345-352f-4c5e-b400-56dbc0546501-cert\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.114606 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.114590 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f58d8345-352f-4c5e-b400-56dbc0546501-manager-config\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.116893 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.116871 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f58d8345-352f-4c5e-b400-56dbc0546501-metrics-cert\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.117028 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.117008 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f58d8345-352f-4c5e-b400-56dbc0546501-cert\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.128697 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.128672 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx48n\" (UniqueName: \"kubernetes.io/projected/f58d8345-352f-4c5e-b400-56dbc0546501-kube-api-access-dx48n\") pod \"lws-controller-manager-755b658494-fj45k\" (UID: \"f58d8345-352f-4c5e-b400-56dbc0546501\") " pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.170742 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.170711 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:44.374407 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.374369 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-755b658494-fj45k"] Jul 22 11:54:44.376608 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:54:44.376582 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58d8345_352f_4c5e_b400_56dbc0546501.slice/crio-ddba261509ed9ce0ca3aab08e094aa92aef84ab3d7bb642a4b42adda4caacaa7 WatchSource:0}: Error finding container ddba261509ed9ce0ca3aab08e094aa92aef84ab3d7bb642a4b42adda4caacaa7: Status 404 returned error can't find the container with id ddba261509ed9ce0ca3aab08e094aa92aef84ab3d7bb642a4b42adda4caacaa7 Jul 22 11:54:44.378500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.378484 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 11:54:44.708536 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:44.708452 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" event={"ID":"f58d8345-352f-4c5e-b400-56dbc0546501","Type":"ContainerStarted","Data":"ddba261509ed9ce0ca3aab08e094aa92aef84ab3d7bb642a4b42adda4caacaa7"} Jul 22 11:54:50.737861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:50.737816 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" event={"ID":"f58d8345-352f-4c5e-b400-56dbc0546501","Type":"ContainerStarted","Data":"799bf3b94021cb1e953812cf406d1252191fe2569992f0a5017cca45c88c10d3"} Jul 22 11:54:50.738236 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:50.737879 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:54:50.757988 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:54:50.757926 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" podStartSLOduration=1.5643565320000001 podStartE2EDuration="7.757908334s" totalImagesPullingTime="6.193551802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:54:50.757554992 +0000 UTC m=+319.083402256" watchObservedRunningTime="2026-07-22 11:54:50.757908334 +0000 UTC m=+319.083755598" Jul 22 11:55:01.744963 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:01.744911 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-755b658494-fj45k" Jul 22 11:55:08.205843 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.205812 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247"] Jul 22 11:55:08.214173 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.214148 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.218557 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.218534 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 11:55:08.218692 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.218590 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-plxnd\"" Jul 22 11:55:08.226399 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.226377 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247"] Jul 22 11:55:08.307697 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307661 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.307697 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307699 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d19681-f9d1-42bd-b173-04f59eea7835-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.307973 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307731 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ts8g\" (UniqueName: \"kubernetes.io/projected/54d19681-f9d1-42bd-b173-04f59eea7835-kube-api-access-9ts8g\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.307973 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307873 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.307973 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307907 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d19681-f9d1-42bd-b173-04f59eea7835-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.307973 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307959 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d19681-f9d1-42bd-b173-04f59eea7835-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.308132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.307983 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.308132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.308006 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.308132 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.308105 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409003 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.408967 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409003 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409001 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d19681-f9d1-42bd-b173-04f59eea7835-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409035 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d19681-f9d1-42bd-b173-04f59eea7835-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409060 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409080 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409120 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409160 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409190 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d19681-f9d1-42bd-b173-04f59eea7835-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409231 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409218 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ts8g\" (UniqueName: \"kubernetes.io/projected/54d19681-f9d1-42bd-b173-04f59eea7835-kube-api-access-9ts8g\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409577 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409425 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409577 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409529 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409647 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409630 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409684 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409661 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.409890 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.409866 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d19681-f9d1-42bd-b173-04f59eea7835-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.411434 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.411412 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d19681-f9d1-42bd-b173-04f59eea7835-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.411785 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.411771 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d19681-f9d1-42bd-b173-04f59eea7835-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.422157 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.422138 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ts8g\" (UniqueName: \"kubernetes.io/projected/54d19681-f9d1-42bd-b173-04f59eea7835-kube-api-access-9ts8g\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.422483 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.422464 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d19681-f9d1-42bd-b173-04f59eea7835-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47p247\" (UID: \"54d19681-f9d1-42bd-b173-04f59eea7835\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.523844 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.523738 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:08.725082 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.725007 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247"] Jul 22 11:55:08.727766 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:55:08.727733 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d19681_f9d1_42bd_b173_04f59eea7835.slice/crio-c8d9ed5fdc0e551ad5bfbb8887cbdcc4b521259b124f7d71abf5732ae4bbff04 WatchSource:0}: Error finding container c8d9ed5fdc0e551ad5bfbb8887cbdcc4b521259b124f7d71abf5732ae4bbff04: Status 404 returned error can't find the container with id c8d9ed5fdc0e551ad5bfbb8887cbdcc4b521259b124f7d71abf5732ae4bbff04 Jul 22 11:55:08.827148 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:08.827060 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" event={"ID":"54d19681-f9d1-42bd-b173-04f59eea7835","Type":"ContainerStarted","Data":"c8d9ed5fdc0e551ad5bfbb8887cbdcc4b521259b124f7d71abf5732ae4bbff04"} Jul 22 11:55:11.558704 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:11.558664 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:55:11.559011 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:11.558744 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:55:11.559011 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:11.558772 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:55:11.845978 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:11.845871 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" event={"ID":"54d19681-f9d1-42bd-b173-04f59eea7835","Type":"ContainerStarted","Data":"0f58969f2c2dfc80a102793948748be5f06e198bef51c09d17594cb72d242e2c"} Jul 22 11:55:11.902861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:11.902788 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" podStartSLOduration=1.074049063 podStartE2EDuration="3.902767917s" totalImagesPullingTime="2.828718854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:55:11.901970173 +0000 UTC m=+340.227817438" watchObservedRunningTime="2026-07-22 11:55:11.902767917 +0000 UTC m=+340.228615180" Jul 22 11:55:12.524701 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:12.524623 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:12.529150 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:12.529127 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:12.850597 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:12.850516 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:12.851692 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:12.851669 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47p247" Jul 22 11:55:47.614814 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.614713 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pvhjl"] Jul 22 11:55:47.625661 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.625634 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:47.628694 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.628664 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pvhjl"] Jul 22 11:55:47.629242 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.629222 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-sg8s4\"" Jul 22 11:55:47.629335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.629237 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 11:55:47.629822 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.629777 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 11:55:47.751369 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.751335 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppmzm\" (UniqueName: \"kubernetes.io/projected/86bfa9fd-0d3f-4ca6-9068-20ac564cd058-kube-api-access-ppmzm\") pod \"kuadrant-operator-catalog-pvhjl\" (UID: \"86bfa9fd-0d3f-4ca6-9068-20ac564cd058\") " pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:47.852513 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.852480 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppmzm\" (UniqueName: \"kubernetes.io/projected/86bfa9fd-0d3f-4ca6-9068-20ac564cd058-kube-api-access-ppmzm\") pod \"kuadrant-operator-catalog-pvhjl\" (UID: \"86bfa9fd-0d3f-4ca6-9068-20ac564cd058\") " pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:47.864492 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.864469 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppmzm\" (UniqueName: \"kubernetes.io/projected/86bfa9fd-0d3f-4ca6-9068-20ac564cd058-kube-api-access-ppmzm\") pod \"kuadrant-operator-catalog-pvhjl\" (UID: \"86bfa9fd-0d3f-4ca6-9068-20ac564cd058\") " pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:47.937119 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.937088 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:47.954980 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:47.954957 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pvhjl"] Jul 22 11:55:48.078088 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.078065 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pvhjl"] Jul 22 11:55:48.079826 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:55:48.079784 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86bfa9fd_0d3f_4ca6_9068_20ac564cd058.slice/crio-65a0697c4ce5f2940937da76f7107e1580d6f2ced08d732d88cc7aacdb5d64b2 WatchSource:0}: Error finding container 65a0697c4ce5f2940937da76f7107e1580d6f2ced08d732d88cc7aacdb5d64b2: Status 404 returned error can't find the container with id 65a0697c4ce5f2940937da76f7107e1580d6f2ced08d732d88cc7aacdb5d64b2 Jul 22 11:55:48.166965 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.166934 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w7g4n"] Jul 22 11:55:48.171962 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.171944 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:48.181369 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.181345 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w7g4n"] Jul 22 11:55:48.357024 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.356928 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhfgv\" (UniqueName: \"kubernetes.io/projected/5610964c-edf1-4d86-9e6f-5de894942733-kube-api-access-xhfgv\") pod \"kuadrant-operator-catalog-w7g4n\" (UID: \"5610964c-edf1-4d86-9e6f-5de894942733\") " pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:48.457360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.457327 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhfgv\" (UniqueName: \"kubernetes.io/projected/5610964c-edf1-4d86-9e6f-5de894942733-kube-api-access-xhfgv\") pod \"kuadrant-operator-catalog-w7g4n\" (UID: \"5610964c-edf1-4d86-9e6f-5de894942733\") " pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:48.468925 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.468901 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhfgv\" (UniqueName: \"kubernetes.io/projected/5610964c-edf1-4d86-9e6f-5de894942733-kube-api-access-xhfgv\") pod \"kuadrant-operator-catalog-w7g4n\" (UID: \"5610964c-edf1-4d86-9e6f-5de894942733\") " pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:48.481655 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.481631 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:48.628090 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:48.628066 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w7g4n"] Jul 22 11:55:48.630044 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:55:48.630018 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5610964c_edf1_4d86_9e6f_5de894942733.slice/crio-206d0b83d69aa1e71e2a2ba278b4c44052decc4b0e4a5a61e4962bdf4950db4a WatchSource:0}: Error finding container 206d0b83d69aa1e71e2a2ba278b4c44052decc4b0e4a5a61e4962bdf4950db4a: Status 404 returned error can't find the container with id 206d0b83d69aa1e71e2a2ba278b4c44052decc4b0e4a5a61e4962bdf4950db4a Jul 22 11:55:49.025333 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:49.025246 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" event={"ID":"5610964c-edf1-4d86-9e6f-5de894942733","Type":"ContainerStarted","Data":"206d0b83d69aa1e71e2a2ba278b4c44052decc4b0e4a5a61e4962bdf4950db4a"} Jul 22 11:55:49.026718 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:49.026684 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" event={"ID":"86bfa9fd-0d3f-4ca6-9068-20ac564cd058","Type":"ContainerStarted","Data":"65a0697c4ce5f2940937da76f7107e1580d6f2ced08d732d88cc7aacdb5d64b2"} Jul 22 11:55:51.040683 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.040643 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" event={"ID":"86bfa9fd-0d3f-4ca6-9068-20ac564cd058","Type":"ContainerStarted","Data":"ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742"} Jul 22 11:55:51.040683 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.040645 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" podUID="86bfa9fd-0d3f-4ca6-9068-20ac564cd058" containerName="registry-server" containerID="cri-o://ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742" gracePeriod=2 Jul 22 11:55:51.042425 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.042401 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" event={"ID":"5610964c-edf1-4d86-9e6f-5de894942733","Type":"ContainerStarted","Data":"a3d178d49d4e3d81e8b48ed778045f2ef8167212b9d74f31ee33e889e5c55635"} Jul 22 11:55:51.060641 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.060601 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" podStartSLOduration=1.862094764 podStartE2EDuration="4.060587305s" totalImagesPullingTime="2.198492541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:55:51.05933662 +0000 UTC m=+379.385183885" watchObservedRunningTime="2026-07-22 11:55:51.060587305 +0000 UTC m=+379.386434568" Jul 22 11:55:51.292991 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.292921 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:51.311428 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.311374 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" podStartSLOduration=1.662249531 podStartE2EDuration="3.311354373s" totalImagesPullingTime="1.649104842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:55:51.080117718 +0000 UTC m=+379.405964982" watchObservedRunningTime="2026-07-22 11:55:51.311354373 +0000 UTC m=+379.637201641" Jul 22 11:55:51.484968 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.484927 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppmzm\" (UniqueName: \"kubernetes.io/projected/86bfa9fd-0d3f-4ca6-9068-20ac564cd058-kube-api-access-ppmzm\") pod \"86bfa9fd-0d3f-4ca6-9068-20ac564cd058\" (UID: \"86bfa9fd-0d3f-4ca6-9068-20ac564cd058\") " Jul 22 11:55:51.487183 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.487158 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86bfa9fd-0d3f-4ca6-9068-20ac564cd058-kube-api-access-ppmzm" (OuterVolumeSpecName: "kube-api-access-ppmzm") pod "86bfa9fd-0d3f-4ca6-9068-20ac564cd058" (UID: "86bfa9fd-0d3f-4ca6-9068-20ac564cd058"). InnerVolumeSpecName "kube-api-access-ppmzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:55:51.585918 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:51.585844 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppmzm\" (UniqueName: \"kubernetes.io/projected/86bfa9fd-0d3f-4ca6-9068-20ac564cd058-kube-api-access-ppmzm\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:55:52.048879 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.048845 2534 generic.go:356] "Generic (PLEG): container finished" podID="86bfa9fd-0d3f-4ca6-9068-20ac564cd058" containerID="ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742" exitCode=0 Jul 22 11:55:52.048879 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.048880 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" Jul 22 11:55:52.049433 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.048932 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" event={"ID":"86bfa9fd-0d3f-4ca6-9068-20ac564cd058","Type":"ContainerDied","Data":"ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742"} Jul 22 11:55:52.049433 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.049000 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pvhjl" event={"ID":"86bfa9fd-0d3f-4ca6-9068-20ac564cd058","Type":"ContainerDied","Data":"65a0697c4ce5f2940937da76f7107e1580d6f2ced08d732d88cc7aacdb5d64b2"} Jul 22 11:55:52.049433 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.049030 2534 scope.go:117] "RemoveContainer" containerID="ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742" Jul 22 11:55:52.058296 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.058273 2534 scope.go:117] "RemoveContainer" containerID="ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742" Jul 22 11:55:52.058540 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:55:52.058521 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742\": container with ID starting with ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742 not found: ID does not exist" containerID="ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742" Jul 22 11:55:52.058593 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.058548 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742"} err="failed to get container status \"ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742\": rpc error: code = NotFound desc = could not find container \"ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742\": container with ID starting with ef8679100b28ed8c52fbf5949b58b6db2e904429f4954e735f7fb236c24d7742 not found: ID does not exist" Jul 22 11:55:52.071874 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.071846 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pvhjl"] Jul 22 11:55:52.073529 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.073512 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pvhjl"] Jul 22 11:55:52.100543 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:52.100507 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86bfa9fd-0d3f-4ca6-9068-20ac564cd058" path="/var/lib/kubelet/pods/86bfa9fd-0d3f-4ca6-9068-20ac564cd058/volumes" Jul 22 11:55:58.482091 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:58.482045 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:58.482540 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:58.482107 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:58.504950 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:58.504912 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:55:59.107419 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:55:59.107394 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-w7g4n" Jul 22 11:56:02.765975 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.765937 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz"] Jul 22 11:56:02.766353 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.766277 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86bfa9fd-0d3f-4ca6-9068-20ac564cd058" containerName="registry-server" Jul 22 11:56:02.766353 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.766289 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="86bfa9fd-0d3f-4ca6-9068-20ac564cd058" containerName="registry-server" Jul 22 11:56:02.766420 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.766368 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="86bfa9fd-0d3f-4ca6-9068-20ac564cd058" containerName="registry-server" Jul 22 11:56:02.769566 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.769552 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.772458 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.772438 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-z7z59\"" Jul 22 11:56:02.782287 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.782266 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz"] Jul 22 11:56:02.880929 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.880898 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.881101 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.880964 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.881101 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.880982 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghxmt\" (UniqueName: \"kubernetes.io/projected/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-kube-api-access-ghxmt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.981531 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.981500 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.981713 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.981555 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.981713 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.981684 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghxmt\" (UniqueName: \"kubernetes.io/projected/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-kube-api-access-ghxmt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.981915 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.981899 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.981968 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.981950 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:02.992070 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:02.992045 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghxmt\" (UniqueName: \"kubernetes.io/projected/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-kube-api-access-ghxmt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:03.079186 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.079105 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:03.171249 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.171223 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv"] Jul 22 11:56:03.177928 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.177897 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.185885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.185860 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv"] Jul 22 11:56:03.227540 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.227514 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz"] Jul 22 11:56:03.229431 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:56:03.229406 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0389f7_2c3d_41d4_a843_ee045b4dbd87.slice/crio-f038045d666e00bf15ee37fe2db55992395e546c2e1fd3e89d074ff4e45fe91a WatchSource:0}: Error finding container f038045d666e00bf15ee37fe2db55992395e546c2e1fd3e89d074ff4e45fe91a: Status 404 returned error can't find the container with id f038045d666e00bf15ee37fe2db55992395e546c2e1fd3e89d074ff4e45fe91a Jul 22 11:56:03.283575 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.283554 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.283709 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.283603 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.283773 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.283710 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd7rx\" (UniqueName: \"kubernetes.io/projected/82fd35dc-c6ab-490e-83c3-c8af34644bf4-kube-api-access-jd7rx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.385009 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.384981 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.385149 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.385044 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.385149 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.385092 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd7rx\" (UniqueName: \"kubernetes.io/projected/82fd35dc-c6ab-490e-83c3-c8af34644bf4-kube-api-access-jd7rx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.385376 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.385347 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.385490 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.385396 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.395815 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.395778 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd7rx\" (UniqueName: \"kubernetes.io/projected/82fd35dc-c6ab-490e-83c3-c8af34644bf4-kube-api-access-jd7rx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.490271 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.490229 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:03.631482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.631454 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv"] Jul 22 11:56:03.633290 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:56:03.633262 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fd35dc_c6ab_490e_83c3_c8af34644bf4.slice/crio-f606b459cba8368f739d8ba223eb4e5f0e32e646b928aa1db47edaffb8caf05e WatchSource:0}: Error finding container f606b459cba8368f739d8ba223eb4e5f0e32e646b928aa1db47edaffb8caf05e: Status 404 returned error can't find the container with id f606b459cba8368f739d8ba223eb4e5f0e32e646b928aa1db47edaffb8caf05e Jul 22 11:56:03.775234 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.775200 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr"] Jul 22 11:56:03.778534 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.778517 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.789627 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.789600 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr"] Jul 22 11:56:03.889280 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.889203 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.889280 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.889240 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.889443 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.889326 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmtjs\" (UniqueName: \"kubernetes.io/projected/492a79d2-1733-4f8b-8d7e-0be3ce80c189-kube-api-access-lmtjs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.990687 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.990652 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.990865 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.990693 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.990865 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.990734 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtjs\" (UniqueName: \"kubernetes.io/projected/492a79d2-1733-4f8b-8d7e-0be3ce80c189-kube-api-access-lmtjs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.991057 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.991040 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:03.991095 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:03.991068 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:04.001612 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.001593 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtjs\" (UniqueName: \"kubernetes.io/projected/492a79d2-1733-4f8b-8d7e-0be3ce80c189-kube-api-access-lmtjs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:04.087771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.087733 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:04.112263 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.112211 2534 generic.go:356] "Generic (PLEG): container finished" podID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerID="0c9e03f38d21c2a8538af4e1244e1568b92af8d1d78f2e6fb0b9e7ed4da433de" exitCode=0 Jul 22 11:56:04.112593 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.112487 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" event={"ID":"6e0389f7-2c3d-41d4-a843-ee045b4dbd87","Type":"ContainerDied","Data":"0c9e03f38d21c2a8538af4e1244e1568b92af8d1d78f2e6fb0b9e7ed4da433de"} Jul 22 11:56:04.112729 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.112620 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" event={"ID":"6e0389f7-2c3d-41d4-a843-ee045b4dbd87","Type":"ContainerStarted","Data":"f038045d666e00bf15ee37fe2db55992395e546c2e1fd3e89d074ff4e45fe91a"} Jul 22 11:56:04.114696 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.114663 2534 generic.go:356] "Generic (PLEG): container finished" podID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerID="44121422370b30db8d6815613b5ae0cd784180f1b33f7f079f2022bed8f141fd" exitCode=0 Jul 22 11:56:04.114851 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.114742 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" event={"ID":"82fd35dc-c6ab-490e-83c3-c8af34644bf4","Type":"ContainerDied","Data":"44121422370b30db8d6815613b5ae0cd784180f1b33f7f079f2022bed8f141fd"} Jul 22 11:56:04.114851 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.114807 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" event={"ID":"82fd35dc-c6ab-490e-83c3-c8af34644bf4","Type":"ContainerStarted","Data":"f606b459cba8368f739d8ba223eb4e5f0e32e646b928aa1db47edaffb8caf05e"} Jul 22 11:56:04.236904 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.236868 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr"] Jul 22 11:56:04.239194 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:56:04.239171 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492a79d2_1733_4f8b_8d7e_0be3ce80c189.slice/crio-7a3a806791575ecfe2b3bc3a2491fbf28fa12511610c6f0f7b2e48dad6e83b02 WatchSource:0}: Error finding container 7a3a806791575ecfe2b3bc3a2491fbf28fa12511610c6f0f7b2e48dad6e83b02: Status 404 returned error can't find the container with id 7a3a806791575ecfe2b3bc3a2491fbf28fa12511610c6f0f7b2e48dad6e83b02 Jul 22 11:56:04.353063 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.353036 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx"] Jul 22 11:56:04.356562 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.356542 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.366360 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.366341 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx"] Jul 22 11:56:04.395143 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.395075 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szvgd\" (UniqueName: \"kubernetes.io/projected/1a284ee5-e8f7-4005-8a34-80ff91f35254-kube-api-access-szvgd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.395143 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.395130 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.395295 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.395179 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.495631 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.495599 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.495758 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.495636 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.495758 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.495713 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szvgd\" (UniqueName: \"kubernetes.io/projected/1a284ee5-e8f7-4005-8a34-80ff91f35254-kube-api-access-szvgd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.496096 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.496078 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.496133 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.496073 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.510481 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.510451 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szvgd\" (UniqueName: \"kubernetes.io/projected/1a284ee5-e8f7-4005-8a34-80ff91f35254-kube-api-access-szvgd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.720936 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.720849 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:04.879499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:04.879465 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx"] Jul 22 11:56:04.940548 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:56:04.940512 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a284ee5_e8f7_4005_8a34_80ff91f35254.slice/crio-8f490abcb63d169642b7a7cd8f09a2ef65b6de1dccf335059fb26c4a808b5215 WatchSource:0}: Error finding container 8f490abcb63d169642b7a7cd8f09a2ef65b6de1dccf335059fb26c4a808b5215: Status 404 returned error can't find the container with id 8f490abcb63d169642b7a7cd8f09a2ef65b6de1dccf335059fb26c4a808b5215 Jul 22 11:56:05.121658 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.121630 2534 generic.go:356] "Generic (PLEG): container finished" podID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerID="a6ac4aed77ed3ebf0b7e562a9423e3121ece67ff4030fe6d12a6a8081281dbf8" exitCode=0 Jul 22 11:56:05.121763 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.121711 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" event={"ID":"1a284ee5-e8f7-4005-8a34-80ff91f35254","Type":"ContainerDied","Data":"a6ac4aed77ed3ebf0b7e562a9423e3121ece67ff4030fe6d12a6a8081281dbf8"} Jul 22 11:56:05.121852 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.121764 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" event={"ID":"1a284ee5-e8f7-4005-8a34-80ff91f35254","Type":"ContainerStarted","Data":"8f490abcb63d169642b7a7cd8f09a2ef65b6de1dccf335059fb26c4a808b5215"} Jul 22 11:56:05.123471 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.123443 2534 generic.go:356] "Generic (PLEG): container finished" podID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerID="885919526d236688426d70c230899d8932a75ec85de792951e8ce0705578c6ec" exitCode=0 Jul 22 11:56:05.123608 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.123585 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" event={"ID":"492a79d2-1733-4f8b-8d7e-0be3ce80c189","Type":"ContainerDied","Data":"885919526d236688426d70c230899d8932a75ec85de792951e8ce0705578c6ec"} Jul 22 11:56:05.123695 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.123640 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" event={"ID":"492a79d2-1733-4f8b-8d7e-0be3ce80c189","Type":"ContainerStarted","Data":"7a3a806791575ecfe2b3bc3a2491fbf28fa12511610c6f0f7b2e48dad6e83b02"} Jul 22 11:56:05.125709 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.125684 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" event={"ID":"6e0389f7-2c3d-41d4-a843-ee045b4dbd87","Type":"ContainerStarted","Data":"681952fe469710dc9949dc05f3c02d4d5cebe18a16586e33eebb9ffdc05c48b3"} Jul 22 11:56:05.128164 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.128138 2534 generic.go:356] "Generic (PLEG): container finished" podID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerID="5f08e351b7eb9388da4b926868f17814e108282addfcb2ee1cf5a4052a6c8d8d" exitCode=0 Jul 22 11:56:05.128419 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:05.128179 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" event={"ID":"82fd35dc-c6ab-490e-83c3-c8af34644bf4","Type":"ContainerDied","Data":"5f08e351b7eb9388da4b926868f17814e108282addfcb2ee1cf5a4052a6c8d8d"} Jul 22 11:56:06.135271 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.135240 2534 generic.go:356] "Generic (PLEG): container finished" podID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerID="11ec8ea527374083c70dba674c42a9a56e7df9c71f3bb5098024c18504bbf259" exitCode=0 Jul 22 11:56:06.135665 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.135279 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" event={"ID":"1a284ee5-e8f7-4005-8a34-80ff91f35254","Type":"ContainerDied","Data":"11ec8ea527374083c70dba674c42a9a56e7df9c71f3bb5098024c18504bbf259"} Jul 22 11:56:06.137336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.137314 2534 generic.go:356] "Generic (PLEG): container finished" podID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerID="a520730d007de06d65c1f768b854ee51e3a6958b4bb22c234d4bceda89bf8768" exitCode=0 Jul 22 11:56:06.137448 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.137410 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" event={"ID":"492a79d2-1733-4f8b-8d7e-0be3ce80c189","Type":"ContainerDied","Data":"a520730d007de06d65c1f768b854ee51e3a6958b4bb22c234d4bceda89bf8768"} Jul 22 11:56:06.139335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.139315 2534 generic.go:356] "Generic (PLEG): container finished" podID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerID="681952fe469710dc9949dc05f3c02d4d5cebe18a16586e33eebb9ffdc05c48b3" exitCode=0 Jul 22 11:56:06.139408 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.139385 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" event={"ID":"6e0389f7-2c3d-41d4-a843-ee045b4dbd87","Type":"ContainerDied","Data":"681952fe469710dc9949dc05f3c02d4d5cebe18a16586e33eebb9ffdc05c48b3"} Jul 22 11:56:06.141608 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.141586 2534 generic.go:356] "Generic (PLEG): container finished" podID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerID="a6cae19b2a2604eb00cd7970d52bc3de8387ec378024212e4afbf5f7d9a3471e" exitCode=0 Jul 22 11:56:06.141686 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:06.141634 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" event={"ID":"82fd35dc-c6ab-490e-83c3-c8af34644bf4","Type":"ContainerDied","Data":"a6cae19b2a2604eb00cd7970d52bc3de8387ec378024212e4afbf5f7d9a3471e"} Jul 22 11:56:07.154400 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.154360 2534 generic.go:356] "Generic (PLEG): container finished" podID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerID="7277cc4891cae6dbe38bbc0b8399d16e4c4c321243b810c05bc9cfc0c11d736a" exitCode=0 Jul 22 11:56:07.154842 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.154443 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" event={"ID":"1a284ee5-e8f7-4005-8a34-80ff91f35254","Type":"ContainerDied","Data":"7277cc4891cae6dbe38bbc0b8399d16e4c4c321243b810c05bc9cfc0c11d736a"} Jul 22 11:56:07.156548 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.156526 2534 generic.go:356] "Generic (PLEG): container finished" podID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerID="d9265707e1f5095bea458f02c1851e98553541700cbc42345678943fb059e945" exitCode=0 Jul 22 11:56:07.156637 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.156614 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" event={"ID":"492a79d2-1733-4f8b-8d7e-0be3ce80c189","Type":"ContainerDied","Data":"d9265707e1f5095bea458f02c1851e98553541700cbc42345678943fb059e945"} Jul 22 11:56:07.158625 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.158594 2534 generic.go:356] "Generic (PLEG): container finished" podID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerID="f8933f1c2ee302e78127d4c755947330e937eaeacbfd0e7dc494abb05fb241a8" exitCode=0 Jul 22 11:56:07.158729 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.158624 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" event={"ID":"6e0389f7-2c3d-41d4-a843-ee045b4dbd87","Type":"ContainerDied","Data":"f8933f1c2ee302e78127d4c755947330e937eaeacbfd0e7dc494abb05fb241a8"} Jul 22 11:56:07.292498 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.292474 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:07.423092 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.423054 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-bundle\") pod \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " Jul 22 11:56:07.423092 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.423098 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-util\") pod \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " Jul 22 11:56:07.423345 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.423115 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd7rx\" (UniqueName: \"kubernetes.io/projected/82fd35dc-c6ab-490e-83c3-c8af34644bf4-kube-api-access-jd7rx\") pod \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\" (UID: \"82fd35dc-c6ab-490e-83c3-c8af34644bf4\") " Jul 22 11:56:07.423655 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.423631 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-bundle" (OuterVolumeSpecName: "bundle") pod "82fd35dc-c6ab-490e-83c3-c8af34644bf4" (UID: "82fd35dc-c6ab-490e-83c3-c8af34644bf4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:07.425479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.425458 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82fd35dc-c6ab-490e-83c3-c8af34644bf4-kube-api-access-jd7rx" (OuterVolumeSpecName: "kube-api-access-jd7rx") pod "82fd35dc-c6ab-490e-83c3-c8af34644bf4" (UID: "82fd35dc-c6ab-490e-83c3-c8af34644bf4"). InnerVolumeSpecName "kube-api-access-jd7rx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:56:07.428182 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.428163 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-util" (OuterVolumeSpecName: "util") pod "82fd35dc-c6ab-490e-83c3-c8af34644bf4" (UID: "82fd35dc-c6ab-490e-83c3-c8af34644bf4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:07.524408 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.524304 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:07.524408 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.524350 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82fd35dc-c6ab-490e-83c3-c8af34644bf4-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:07.524408 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:07.524359 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd7rx\" (UniqueName: \"kubernetes.io/projected/82fd35dc-c6ab-490e-83c3-c8af34644bf4-kube-api-access-jd7rx\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.165336 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.165302 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" event={"ID":"82fd35dc-c6ab-490e-83c3-c8af34644bf4","Type":"ContainerDied","Data":"f606b459cba8368f739d8ba223eb4e5f0e32e646b928aa1db47edaffb8caf05e"} Jul 22 11:56:08.165697 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.165358 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f606b459cba8368f739d8ba223eb4e5f0e32e646b928aa1db47edaffb8caf05e" Jul 22 11:56:08.165697 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.165313 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv" Jul 22 11:56:08.317514 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.317489 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:08.359341 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.359277 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:08.362209 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.362189 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:08.435158 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435125 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmtjs\" (UniqueName: \"kubernetes.io/projected/492a79d2-1733-4f8b-8d7e-0be3ce80c189-kube-api-access-lmtjs\") pod \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " Jul 22 11:56:08.435296 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435176 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-bundle\") pod \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " Jul 22 11:56:08.435296 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435206 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szvgd\" (UniqueName: \"kubernetes.io/projected/1a284ee5-e8f7-4005-8a34-80ff91f35254-kube-api-access-szvgd\") pod \"1a284ee5-e8f7-4005-8a34-80ff91f35254\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " Jul 22 11:56:08.435296 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435240 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-bundle\") pod \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " Jul 22 11:56:08.435296 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435259 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-util\") pod \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\" (UID: \"492a79d2-1733-4f8b-8d7e-0be3ce80c189\") " Jul 22 11:56:08.435296 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435287 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-bundle\") pod \"1a284ee5-e8f7-4005-8a34-80ff91f35254\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " Jul 22 11:56:08.435551 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435324 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-util\") pod \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " Jul 22 11:56:08.435551 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435352 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-util\") pod \"1a284ee5-e8f7-4005-8a34-80ff91f35254\" (UID: \"1a284ee5-e8f7-4005-8a34-80ff91f35254\") " Jul 22 11:56:08.435551 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435416 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghxmt\" (UniqueName: \"kubernetes.io/projected/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-kube-api-access-ghxmt\") pod \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\" (UID: \"6e0389f7-2c3d-41d4-a843-ee045b4dbd87\") " Jul 22 11:56:08.436206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.435844 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-bundle" (OuterVolumeSpecName: "bundle") pod "1a284ee5-e8f7-4005-8a34-80ff91f35254" (UID: "1a284ee5-e8f7-4005-8a34-80ff91f35254"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:08.436206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.436076 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-bundle" (OuterVolumeSpecName: "bundle") pod "492a79d2-1733-4f8b-8d7e-0be3ce80c189" (UID: "492a79d2-1733-4f8b-8d7e-0be3ce80c189"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:08.436512 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.436488 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-bundle" (OuterVolumeSpecName: "bundle") pod "6e0389f7-2c3d-41d4-a843-ee045b4dbd87" (UID: "6e0389f7-2c3d-41d4-a843-ee045b4dbd87"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:08.437933 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.437906 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492a79d2-1733-4f8b-8d7e-0be3ce80c189-kube-api-access-lmtjs" (OuterVolumeSpecName: "kube-api-access-lmtjs") pod "492a79d2-1733-4f8b-8d7e-0be3ce80c189" (UID: "492a79d2-1733-4f8b-8d7e-0be3ce80c189"). InnerVolumeSpecName "kube-api-access-lmtjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:56:08.438053 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.437942 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a284ee5-e8f7-4005-8a34-80ff91f35254-kube-api-access-szvgd" (OuterVolumeSpecName: "kube-api-access-szvgd") pod "1a284ee5-e8f7-4005-8a34-80ff91f35254" (UID: "1a284ee5-e8f7-4005-8a34-80ff91f35254"). InnerVolumeSpecName "kube-api-access-szvgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:56:08.438208 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.438181 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-kube-api-access-ghxmt" (OuterVolumeSpecName: "kube-api-access-ghxmt") pod "6e0389f7-2c3d-41d4-a843-ee045b4dbd87" (UID: "6e0389f7-2c3d-41d4-a843-ee045b4dbd87"). InnerVolumeSpecName "kube-api-access-ghxmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:56:08.441268 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.441247 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-util" (OuterVolumeSpecName: "util") pod "492a79d2-1733-4f8b-8d7e-0be3ce80c189" (UID: "492a79d2-1733-4f8b-8d7e-0be3ce80c189"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:08.441936 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.441911 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-util" (OuterVolumeSpecName: "util") pod "6e0389f7-2c3d-41d4-a843-ee045b4dbd87" (UID: "6e0389f7-2c3d-41d4-a843-ee045b4dbd87"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:08.442561 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.442542 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-util" (OuterVolumeSpecName: "util") pod "1a284ee5-e8f7-4005-8a34-80ff91f35254" (UID: "1a284ee5-e8f7-4005-8a34-80ff91f35254"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:56:08.536509 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536474 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536509 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536501 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536509 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536513 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghxmt\" (UniqueName: \"kubernetes.io/projected/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-kube-api-access-ghxmt\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536527 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmtjs\" (UniqueName: \"kubernetes.io/projected/492a79d2-1733-4f8b-8d7e-0be3ce80c189-kube-api-access-lmtjs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536539 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6e0389f7-2c3d-41d4-a843-ee045b4dbd87-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536551 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szvgd\" (UniqueName: \"kubernetes.io/projected/1a284ee5-e8f7-4005-8a34-80ff91f35254-kube-api-access-szvgd\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536561 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536570 2534 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/492a79d2-1733-4f8b-8d7e-0be3ce80c189-util\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:08.536739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:08.536580 2534 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a284ee5-e8f7-4005-8a34-80ff91f35254-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:56:09.171874 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.171842 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" Jul 22 11:56:09.171874 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.171854 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr" event={"ID":"492a79d2-1733-4f8b-8d7e-0be3ce80c189","Type":"ContainerDied","Data":"7a3a806791575ecfe2b3bc3a2491fbf28fa12511610c6f0f7b2e48dad6e83b02"} Jul 22 11:56:09.172343 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.171910 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a3a806791575ecfe2b3bc3a2491fbf28fa12511610c6f0f7b2e48dad6e83b02" Jul 22 11:56:09.173880 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.173858 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" Jul 22 11:56:09.173880 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.173865 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz" event={"ID":"6e0389f7-2c3d-41d4-a843-ee045b4dbd87","Type":"ContainerDied","Data":"f038045d666e00bf15ee37fe2db55992395e546c2e1fd3e89d074ff4e45fe91a"} Jul 22 11:56:09.174036 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.173906 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f038045d666e00bf15ee37fe2db55992395e546c2e1fd3e89d074ff4e45fe91a" Jul 22 11:56:09.175880 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.175859 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" event={"ID":"1a284ee5-e8f7-4005-8a34-80ff91f35254","Type":"ContainerDied","Data":"8f490abcb63d169642b7a7cd8f09a2ef65b6de1dccf335059fb26c4a808b5215"} Jul 22 11:56:09.175880 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.175872 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx" Jul 22 11:56:09.176050 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:09.175892 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f490abcb63d169642b7a7cd8f09a2ef65b6de1dccf335059fb26c4a808b5215" Jul 22 11:56:15.613525 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613491 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf"] Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613818 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="extract" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613829 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="extract" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613839 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="extract" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613845 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="extract" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613852 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="pull" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613859 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="pull" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613866 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="util" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613871 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="util" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613877 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="extract" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613882 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="extract" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613890 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="util" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613895 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="util" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613902 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="pull" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613907 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="pull" Jul 22 11:56:15.613916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613922 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="util" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613927 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="util" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613933 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="util" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613938 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="util" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613943 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="pull" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613947 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="pull" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613954 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="pull" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613959 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="pull" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613966 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="extract" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.613971 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="extract" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.614020 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e0389f7-2c3d-41d4-a843-ee045b4dbd87" containerName="extract" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.614027 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="82fd35dc-c6ab-490e-83c3-c8af34644bf4" containerName="extract" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.614033 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a284ee5-e8f7-4005-8a34-80ff91f35254" containerName="extract" Jul 22 11:56:15.614322 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.614038 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="492a79d2-1733-4f8b-8d7e-0be3ce80c189" containerName="extract" Jul 22 11:56:15.618291 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.618274 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.621306 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.621286 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pg7tg\"" Jul 22 11:56:15.639436 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.639402 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf"] Jul 22 11:56:15.699021 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.698991 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04c8e043-f323-4867-9ec5-d15d43dbf4d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.699180 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.699076 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnzgq\" (UniqueName: \"kubernetes.io/projected/04c8e043-f323-4867-9ec5-d15d43dbf4d9-kube-api-access-pnzgq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.800195 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.800157 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnzgq\" (UniqueName: \"kubernetes.io/projected/04c8e043-f323-4867-9ec5-d15d43dbf4d9-kube-api-access-pnzgq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.800369 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.800225 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04c8e043-f323-4867-9ec5-d15d43dbf4d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.800553 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.800528 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04c8e043-f323-4867-9ec5-d15d43dbf4d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.814012 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.813985 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnzgq\" (UniqueName: \"kubernetes.io/projected/04c8e043-f323-4867-9ec5-d15d43dbf4d9-kube-api-access-pnzgq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:15.932081 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:15.932047 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:16.093019 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:16.092995 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf"] Jul 22 11:56:16.094305 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:56:16.094279 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c8e043_f323_4867_9ec5_d15d43dbf4d9.slice/crio-1fb494c7f0e98cdd9efa8a2c300cc42a6acdfcec153d1902cbd36ab85a7fcd40 WatchSource:0}: Error finding container 1fb494c7f0e98cdd9efa8a2c300cc42a6acdfcec153d1902cbd36ab85a7fcd40: Status 404 returned error can't find the container with id 1fb494c7f0e98cdd9efa8a2c300cc42a6acdfcec153d1902cbd36ab85a7fcd40 Jul 22 11:56:16.211763 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:16.211687 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" event={"ID":"04c8e043-f323-4867-9ec5-d15d43dbf4d9","Type":"ContainerStarted","Data":"1fb494c7f0e98cdd9efa8a2c300cc42a6acdfcec153d1902cbd36ab85a7fcd40"} Jul 22 11:56:17.344844 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.344812 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx"] Jul 22 11:56:17.349469 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.349453 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:17.353060 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.353043 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5sctf\"" Jul 22 11:56:17.364088 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.364065 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx"] Jul 22 11:56:17.415621 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.415592 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75x8t\" (UniqueName: \"kubernetes.io/projected/7cadb7b1-b5d3-426b-ac26-333e6e8da006-kube-api-access-75x8t\") pod \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" (UID: \"7cadb7b1-b5d3-426b-ac26-333e6e8da006\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:17.517123 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.517083 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75x8t\" (UniqueName: \"kubernetes.io/projected/7cadb7b1-b5d3-426b-ac26-333e6e8da006-kube-api-access-75x8t\") pod \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" (UID: \"7cadb7b1-b5d3-426b-ac26-333e6e8da006\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:17.531299 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.531272 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75x8t\" (UniqueName: \"kubernetes.io/projected/7cadb7b1-b5d3-426b-ac26-333e6e8da006-kube-api-access-75x8t\") pod \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" (UID: \"7cadb7b1-b5d3-426b-ac26-333e6e8da006\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:17.659365 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.659335 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:17.813510 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:17.813487 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx"] Jul 22 11:56:17.815627 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:56:17.815600 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cadb7b1_b5d3_426b_ac26_333e6e8da006.slice/crio-cc84f4750f1359962212f617f4f05e55b89e7d677db6c6344a7ccb21a3b27e75 WatchSource:0}: Error finding container cc84f4750f1359962212f617f4f05e55b89e7d677db6c6344a7ccb21a3b27e75: Status 404 returned error can't find the container with id cc84f4750f1359962212f617f4f05e55b89e7d677db6c6344a7ccb21a3b27e75 Jul 22 11:56:18.223963 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:18.223929 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" event={"ID":"7cadb7b1-b5d3-426b-ac26-333e6e8da006","Type":"ContainerStarted","Data":"cc84f4750f1359962212f617f4f05e55b89e7d677db6c6344a7ccb21a3b27e75"} Jul 22 11:56:24.255731 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:24.255694 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" event={"ID":"7cadb7b1-b5d3-426b-ac26-333e6e8da006","Type":"ContainerStarted","Data":"b94757cc571b5b40fb71e464880db1da39acc300cb3b9dea8ff1a11ead417b8b"} Jul 22 11:56:24.256257 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:24.255833 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:24.284465 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:24.284403 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" podStartSLOduration=1.449916587 podStartE2EDuration="7.284389248s" totalImagesPullingTime="5.834472661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:56:24.283217722 +0000 UTC m=+412.609064987" watchObservedRunningTime="2026-07-22 11:56:24.284389248 +0000 UTC m=+412.610236579" Jul 22 11:56:35.263058 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:35.263025 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:56:45.366731 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:45.366696 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" event={"ID":"04c8e043-f323-4867-9ec5-d15d43dbf4d9","Type":"ContainerStarted","Data":"b0073b960b87315f1f4711a1d9c1cd6270c4d1ad9a6a2d04eb0138e920ae2be2"} Jul 22 11:56:45.367149 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:45.366836 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:56:45.438947 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:45.438895 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" podStartSLOduration=1.976476195 podStartE2EDuration="30.438877882s" totalImagesPullingTime="28.462401687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:56:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:56:45.437165264 +0000 UTC m=+433.763012528" watchObservedRunningTime="2026-07-22 11:56:45.438877882 +0000 UTC m=+433.764725463" Jul 22 11:56:56.376342 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:56:56.376257 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.279076 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.279039 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf"] Jul 22 11:57:03.279539 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.279233 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" containerName="manager" containerID="cri-o://b0073b960b87315f1f4711a1d9c1cd6270c4d1ad9a6a2d04eb0138e920ae2be2" gracePeriod=2 Jul 22 11:57:03.287391 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.287336 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf"] Jul 22 11:57:03.313712 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.313686 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx"] Jul 22 11:57:03.314011 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.313964 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" containerName="manager" containerID="cri-o://b94757cc571b5b40fb71e464880db1da39acc300cb3b9dea8ff1a11ead417b8b" gracePeriod=2 Jul 22 11:57:03.350197 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.350166 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx"] Jul 22 11:57:03.362266 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.362233 2534 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7cadb7b1-b5d3-426b-ac26-333e6e8da006\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-j4jdx\": pods \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:03.379944 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.379916 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6"] Jul 22 11:57:03.380266 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.380247 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" containerName="manager" Jul 22 11:57:03.380266 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.380265 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" containerName="manager" Jul 22 11:57:03.380460 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.380289 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" containerName="manager" Jul 22 11:57:03.380460 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.380298 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" containerName="manager" Jul 22 11:57:03.380460 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.380371 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" containerName="manager" Jul 22 11:57:03.380460 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.380388 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" containerName="manager" Jul 22 11:57:03.383615 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.383574 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.384646 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.384624 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx"] Jul 22 11:57:03.384771 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.384758 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.388750 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.388729 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:03.389535 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.389504 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.396450 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.396422 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:03.407932 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.407904 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx"] Jul 22 11:57:03.422984 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.422958 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6"] Jul 22 11:57:03.452204 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.452157 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:03.454965 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.454857 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.468721 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.468682 2534 generic.go:356] "Generic (PLEG): container finished" podID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" containerID="b0073b960b87315f1f4711a1d9c1cd6270c4d1ad9a6a2d04eb0138e920ae2be2" exitCode=0 Jul 22 11:57:03.470471 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.470449 2534 generic.go:356] "Generic (PLEG): container finished" podID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" containerID="b94757cc571b5b40fb71e464880db1da39acc300cb3b9dea8ff1a11ead417b8b" exitCode=0 Jul 22 11:57:03.523382 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.523338 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj7qz\" (UniqueName: \"kubernetes.io/projected/232d9611-9379-4f3a-81e6-7ee2011ccb6f-kube-api-access-vj7qz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k96w6\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.523382 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.523375 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqr8q\" (UniqueName: \"kubernetes.io/projected/a802b12d-7bee-4ddc-b034-5dc83866d9a4-kube-api-access-jqr8q\") pod \"limitador-operator-controller-manager-859d7b78bd-wg5xx\" (UID: \"a802b12d-7bee-4ddc-b034-5dc83866d9a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:03.523619 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.523466 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/232d9611-9379-4f3a-81e6-7ee2011ccb6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k96w6\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.571379 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.571352 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.574040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.574023 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:03.576958 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.576923 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:03.579348 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.579326 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.582706 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.582681 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g69mf\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:03.584892 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:03.584872 2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4jdx\" is forbidden: User \"system:node:ip-10-0-143-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-104.ec2.internal' and this object" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:03.624120 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624089 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75x8t\" (UniqueName: \"kubernetes.io/projected/7cadb7b1-b5d3-426b-ac26-333e6e8da006-kube-api-access-75x8t\") pod \"7cadb7b1-b5d3-426b-ac26-333e6e8da006\" (UID: \"7cadb7b1-b5d3-426b-ac26-333e6e8da006\") " Jul 22 11:57:03.624257 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624133 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnzgq\" (UniqueName: \"kubernetes.io/projected/04c8e043-f323-4867-9ec5-d15d43dbf4d9-kube-api-access-pnzgq\") pod \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " Jul 22 11:57:03.624257 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624194 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04c8e043-f323-4867-9ec5-d15d43dbf4d9-extensions-socket-volume\") pod \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\" (UID: \"04c8e043-f323-4867-9ec5-d15d43dbf4d9\") " Jul 22 11:57:03.624363 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624305 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqr8q\" (UniqueName: \"kubernetes.io/projected/a802b12d-7bee-4ddc-b034-5dc83866d9a4-kube-api-access-jqr8q\") pod \"limitador-operator-controller-manager-859d7b78bd-wg5xx\" (UID: \"a802b12d-7bee-4ddc-b034-5dc83866d9a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:03.624419 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624395 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/232d9611-9379-4f3a-81e6-7ee2011ccb6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k96w6\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.624490 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624471 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj7qz\" (UniqueName: \"kubernetes.io/projected/232d9611-9379-4f3a-81e6-7ee2011ccb6f-kube-api-access-vj7qz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k96w6\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.624770 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624748 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/232d9611-9379-4f3a-81e6-7ee2011ccb6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k96w6\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.624882 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.624824 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c8e043-f323-4867-9ec5-d15d43dbf4d9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "04c8e043-f323-4867-9ec5-d15d43dbf4d9" (UID: "04c8e043-f323-4867-9ec5-d15d43dbf4d9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:03.626532 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.626503 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cadb7b1-b5d3-426b-ac26-333e6e8da006-kube-api-access-75x8t" (OuterVolumeSpecName: "kube-api-access-75x8t") pod "7cadb7b1-b5d3-426b-ac26-333e6e8da006" (UID: "7cadb7b1-b5d3-426b-ac26-333e6e8da006"). InnerVolumeSpecName "kube-api-access-75x8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:57:03.626532 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.626525 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c8e043-f323-4867-9ec5-d15d43dbf4d9-kube-api-access-pnzgq" (OuterVolumeSpecName: "kube-api-access-pnzgq") pod "04c8e043-f323-4867-9ec5-d15d43dbf4d9" (UID: "04c8e043-f323-4867-9ec5-d15d43dbf4d9"). InnerVolumeSpecName "kube-api-access-pnzgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:57:03.644107 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.644083 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqr8q\" (UniqueName: \"kubernetes.io/projected/a802b12d-7bee-4ddc-b034-5dc83866d9a4-kube-api-access-jqr8q\") pod \"limitador-operator-controller-manager-859d7b78bd-wg5xx\" (UID: \"a802b12d-7bee-4ddc-b034-5dc83866d9a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:03.644883 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.644864 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj7qz\" (UniqueName: \"kubernetes.io/projected/232d9611-9379-4f3a-81e6-7ee2011ccb6f-kube-api-access-vj7qz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-k96w6\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.718520 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.718488 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:03.725330 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.725309 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75x8t\" (UniqueName: \"kubernetes.io/projected/7cadb7b1-b5d3-426b-ac26-333e6e8da006-kube-api-access-75x8t\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:57:03.725330 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.725331 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnzgq\" (UniqueName: \"kubernetes.io/projected/04c8e043-f323-4867-9ec5-d15d43dbf4d9-kube-api-access-pnzgq\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:57:03.725504 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.725343 2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04c8e043-f323-4867-9ec5-d15d43dbf4d9-extensions-socket-volume\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:57:03.727219 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.727201 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:03.889257 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.889122 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6"] Jul 22 11:57:03.892311 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:57:03.892281 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232d9611_9379_4f3a_81e6_7ee2011ccb6f.slice/crio-c7fab30ab0911a422d9dc685804d64720dd100677c953f6a27a8a6195cd0c1b1 WatchSource:0}: Error finding container c7fab30ab0911a422d9dc685804d64720dd100677c953f6a27a8a6195cd0c1b1: Status 404 returned error can't find the container with id c7fab30ab0911a422d9dc685804d64720dd100677c953f6a27a8a6195cd0c1b1 Jul 22 11:57:03.895951 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:03.895927 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx"] Jul 22 11:57:03.896158 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:57:03.896138 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda802b12d_7bee_4ddc_b034_5dc83866d9a4.slice/crio-10f4a1f92c1de4cfb7f540ca734b236f1f176e6b8dc6bf7d098e833251a8b278 WatchSource:0}: Error finding container 10f4a1f92c1de4cfb7f540ca734b236f1f176e6b8dc6bf7d098e833251a8b278: Status 404 returned error can't find the container with id 10f4a1f92c1de4cfb7f540ca734b236f1f176e6b8dc6bf7d098e833251a8b278 Jul 22 11:57:04.101207 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.101124 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04c8e043-f323-4867-9ec5-d15d43dbf4d9" path="/var/lib/kubelet/pods/04c8e043-f323-4867-9ec5-d15d43dbf4d9/volumes" Jul 22 11:57:04.101454 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.101441 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cadb7b1-b5d3-426b-ac26-333e6e8da006" path="/var/lib/kubelet/pods/7cadb7b1-b5d3-426b-ac26-333e6e8da006/volumes" Jul 22 11:57:04.476849 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.476820 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g69mf" Jul 22 11:57:04.476849 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.476841 2534 scope.go:117] "RemoveContainer" containerID="b0073b960b87315f1f4711a1d9c1cd6270c4d1ad9a6a2d04eb0138e920ae2be2" Jul 22 11:57:04.478548 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.478524 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4jdx" Jul 22 11:57:04.480525 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.480494 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" event={"ID":"232d9611-9379-4f3a-81e6-7ee2011ccb6f","Type":"ContainerStarted","Data":"ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84"} Jul 22 11:57:04.480661 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.480554 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" event={"ID":"232d9611-9379-4f3a-81e6-7ee2011ccb6f","Type":"ContainerStarted","Data":"c7fab30ab0911a422d9dc685804d64720dd100677c953f6a27a8a6195cd0c1b1"} Jul 22 11:57:04.480817 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.480778 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:04.482333 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.482300 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" event={"ID":"a802b12d-7bee-4ddc-b034-5dc83866d9a4","Type":"ContainerStarted","Data":"f0dd78de80dcad74b54e5475b297e60a4742f519b42ce4b6c7e1fc5236d47129"} Jul 22 11:57:04.482418 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.482339 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" event={"ID":"a802b12d-7bee-4ddc-b034-5dc83866d9a4","Type":"ContainerStarted","Data":"10f4a1f92c1de4cfb7f540ca734b236f1f176e6b8dc6bf7d098e833251a8b278"} Jul 22 11:57:04.482418 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.482369 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:04.486606 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.486590 2534 scope.go:117] "RemoveContainer" containerID="b94757cc571b5b40fb71e464880db1da39acc300cb3b9dea8ff1a11ead417b8b" Jul 22 11:57:04.521411 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:04.521367 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" podStartSLOduration=1.521351075 podStartE2EDuration="1.521351075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:04.520016175 +0000 UTC m=+452.845863443" watchObservedRunningTime="2026-07-22 11:57:04.521351075 +0000 UTC m=+452.847198340" Jul 22 11:57:15.492389 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:15.492356 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:15.492849 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:15.492431 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" Jul 22 11:57:15.515195 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:15.515146 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wg5xx" podStartSLOduration=12.515130470999999 podStartE2EDuration="12.515130471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:04.578990733 +0000 UTC m=+452.904837999" watchObservedRunningTime="2026-07-22 11:57:15.515130471 +0000 UTC m=+463.840977735" Jul 22 11:57:15.915706 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:15.913202 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz"] Jul 22 11:57:15.920176 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:15.920148 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:15.932976 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:15.932951 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz"] Jul 22 11:57:16.040855 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.040756 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e46a5732-7a73-4da7-97a7-e907a87264b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzqtz\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.040855 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.040838 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fff6\" (UniqueName: \"kubernetes.io/projected/e46a5732-7a73-4da7-97a7-e907a87264b6-kube-api-access-6fff6\") pod \"kuadrant-operator-controller-manager-86878c769f-qzqtz\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.141278 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.141244 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e46a5732-7a73-4da7-97a7-e907a87264b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzqtz\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.141437 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.141302 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fff6\" (UniqueName: \"kubernetes.io/projected/e46a5732-7a73-4da7-97a7-e907a87264b6-kube-api-access-6fff6\") pod \"kuadrant-operator-controller-manager-86878c769f-qzqtz\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.141618 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.141585 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e46a5732-7a73-4da7-97a7-e907a87264b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzqtz\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.151624 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.151592 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fff6\" (UniqueName: \"kubernetes.io/projected/e46a5732-7a73-4da7-97a7-e907a87264b6-kube-api-access-6fff6\") pod \"kuadrant-operator-controller-manager-86878c769f-qzqtz\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.229479 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.229448 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.383347 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.383323 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz"] Jul 22 11:57:16.385624 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:57:16.385598 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46a5732_7a73_4da7_97a7_e907a87264b6.slice/crio-35d91cd33a9b73d95942740230b038eecbc7b42587019ab4bdc0579dc5c4b333 WatchSource:0}: Error finding container 35d91cd33a9b73d95942740230b038eecbc7b42587019ab4bdc0579dc5c4b333: Status 404 returned error can't find the container with id 35d91cd33a9b73d95942740230b038eecbc7b42587019ab4bdc0579dc5c4b333 Jul 22 11:57:16.551306 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.551266 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" event={"ID":"e46a5732-7a73-4da7-97a7-e907a87264b6","Type":"ContainerStarted","Data":"475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67"} Jul 22 11:57:16.551774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.551319 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" event={"ID":"e46a5732-7a73-4da7-97a7-e907a87264b6","Type":"ContainerStarted","Data":"35d91cd33a9b73d95942740230b038eecbc7b42587019ab4bdc0579dc5c4b333"} Jul 22 11:57:16.551774 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.551361 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:16.579221 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:16.579142 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" podStartSLOduration=1.579126296 podStartE2EDuration="1.579126296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:16.576977912 +0000 UTC m=+464.902825177" watchObservedRunningTime="2026-07-22 11:57:16.579126296 +0000 UTC m=+464.904973561" Jul 22 11:57:27.559152 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.559115 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 11:57:27.619629 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.619596 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6"] Jul 22 11:57:27.619903 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.619871 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" podUID="232d9611-9379-4f3a-81e6-7ee2011ccb6f" containerName="manager" containerID="cri-o://ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84" gracePeriod=10 Jul 22 11:57:27.880759 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.880733 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:27.942992 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.942958 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/232d9611-9379-4f3a-81e6-7ee2011ccb6f-extensions-socket-volume\") pod \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " Jul 22 11:57:27.943138 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.943000 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj7qz\" (UniqueName: \"kubernetes.io/projected/232d9611-9379-4f3a-81e6-7ee2011ccb6f-kube-api-access-vj7qz\") pod \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\" (UID: \"232d9611-9379-4f3a-81e6-7ee2011ccb6f\") " Jul 22 11:57:27.943323 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.943297 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232d9611-9379-4f3a-81e6-7ee2011ccb6f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "232d9611-9379-4f3a-81e6-7ee2011ccb6f" (UID: "232d9611-9379-4f3a-81e6-7ee2011ccb6f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:27.945206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:27.945184 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/232d9611-9379-4f3a-81e6-7ee2011ccb6f-kube-api-access-vj7qz" (OuterVolumeSpecName: "kube-api-access-vj7qz") pod "232d9611-9379-4f3a-81e6-7ee2011ccb6f" (UID: "232d9611-9379-4f3a-81e6-7ee2011ccb6f"). InnerVolumeSpecName "kube-api-access-vj7qz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:57:28.043932 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.043899 2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/232d9611-9379-4f3a-81e6-7ee2011ccb6f-extensions-socket-volume\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:57:28.043932 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.043925 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj7qz\" (UniqueName: \"kubernetes.io/projected/232d9611-9379-4f3a-81e6-7ee2011ccb6f-kube-api-access-vj7qz\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:57:28.618480 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.618446 2534 generic.go:356] "Generic (PLEG): container finished" podID="232d9611-9379-4f3a-81e6-7ee2011ccb6f" containerID="ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84" exitCode=0 Jul 22 11:57:28.618990 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.618503 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" Jul 22 11:57:28.618990 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.618539 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" event={"ID":"232d9611-9379-4f3a-81e6-7ee2011ccb6f","Type":"ContainerDied","Data":"ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84"} Jul 22 11:57:28.618990 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.618643 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6" event={"ID":"232d9611-9379-4f3a-81e6-7ee2011ccb6f","Type":"ContainerDied","Data":"c7fab30ab0911a422d9dc685804d64720dd100677c953f6a27a8a6195cd0c1b1"} Jul 22 11:57:28.618990 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.618676 2534 scope.go:117] "RemoveContainer" containerID="ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84" Jul 22 11:57:28.628006 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.627815 2534 scope.go:117] "RemoveContainer" containerID="ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84" Jul 22 11:57:28.628113 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:28.628096 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84\": container with ID starting with ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84 not found: ID does not exist" containerID="ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84" Jul 22 11:57:28.628160 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.628122 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84"} err="failed to get container status \"ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84\": rpc error: code = NotFound desc = could not find container \"ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84\": container with ID starting with ae732d7db3e27ce32eb905c541080bc64b230aad7792be94c0afaa280bb05b84 not found: ID does not exist" Jul 22 11:57:28.646959 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.646923 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6"] Jul 22 11:57:28.655695 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:28.655667 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-k96w6"] Jul 22 11:57:30.100966 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:30.100924 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="232d9611-9379-4f3a-81e6-7ee2011ccb6f" path="/var/lib/kubelet/pods/232d9611-9379-4f3a-81e6-7ee2011ccb6f/volumes" Jul 22 11:57:31.892784 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.892747 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq"] Jul 22 11:57:31.893178 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.893124 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="232d9611-9379-4f3a-81e6-7ee2011ccb6f" containerName="manager" Jul 22 11:57:31.893178 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.893136 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="232d9611-9379-4f3a-81e6-7ee2011ccb6f" containerName="manager" Jul 22 11:57:31.893250 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.893202 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="232d9611-9379-4f3a-81e6-7ee2011ccb6f" containerName="manager" Jul 22 11:57:31.897682 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.897664 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.900935 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.900917 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rmhh9\"" Jul 22 11:57:31.914442 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.914421 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq"] Jul 22 11:57:31.974658 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974624 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.974861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974663 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.974861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974737 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.974861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974814 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.974861 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974844 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.975043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974889 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.975043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974911 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gshr\" (UniqueName: \"kubernetes.io/projected/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-kube-api-access-4gshr\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.975043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974926 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:31.975043 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:31.974946 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076170 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076206 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076208 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076431 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076255 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076431 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076299 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076538 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076433 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076538 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076510 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076644 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076547 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gshr\" (UniqueName: \"kubernetes.io/projected/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-kube-api-access-4gshr\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076644 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076575 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076644 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076609 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076824 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076705 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.076824 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.076732 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.077045 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.077023 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.077138 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.077117 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.077215 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.077192 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.078546 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.078526 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.079054 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.079038 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.092373 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.092344 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.093631 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.093606 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gshr\" (UniqueName: \"kubernetes.io/projected/74dd6e20-18d2-49c7-ba3f-26ad91a1c510-kube-api-access-4gshr\") pod \"maas-default-gateway-openshift-default-56654dcf7-9nrnq\" (UID: \"74dd6e20-18d2-49c7-ba3f-26ad91a1c510\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.211434 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.211338 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rmhh9\"" Jul 22 11:57:32.218777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.218757 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:32.368765 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.368737 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq"] Jul 22 11:57:32.370242 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:57:32.370217 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74dd6e20_18d2_49c7_ba3f_26ad91a1c510.slice/crio-55322b4ec5c64e9bd8fefc2a9b3dfa3f88b5845efca9a1b8bb222688bc663f4a WatchSource:0}: Error finding container 55322b4ec5c64e9bd8fefc2a9b3dfa3f88b5845efca9a1b8bb222688bc663f4a: Status 404 returned error can't find the container with id 55322b4ec5c64e9bd8fefc2a9b3dfa3f88b5845efca9a1b8bb222688bc663f4a Jul 22 11:57:32.372373 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.372339 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:32.372459 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.372406 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:32.372459 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.372436 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:32.642223 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.642190 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" event={"ID":"74dd6e20-18d2-49c7-ba3f-26ad91a1c510","Type":"ContainerStarted","Data":"d81635a8627ee4f9b257d689de2be5f73071221b78e55b6f5b76662b74274588"} Jul 22 11:57:32.642412 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.642236 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" event={"ID":"74dd6e20-18d2-49c7-ba3f-26ad91a1c510","Type":"ContainerStarted","Data":"55322b4ec5c64e9bd8fefc2a9b3dfa3f88b5845efca9a1b8bb222688bc663f4a"} Jul 22 11:57:32.670818 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:32.670752 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" podStartSLOduration=1.6707359899999998 podStartE2EDuration="1.67073599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:32.668968564 +0000 UTC m=+480.994815828" watchObservedRunningTime="2026-07-22 11:57:32.67073599 +0000 UTC m=+480.996583253" Jul 22 11:57:33.219695 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:33.219660 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:33.224871 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:33.224850 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:33.648684 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:33.648657 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:33.649469 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:33.649451 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:57:45.214444 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.214411 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-fb5f2"] Jul 22 11:57:45.218933 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.218915 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:45.222215 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.222197 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gd797\"" Jul 22 11:57:45.226202 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.226182 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-fb5f2"] Jul 22 11:57:45.294602 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.294570 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qq2p\" (UniqueName: \"kubernetes.io/projected/dded7869-e070-4b1a-9438-6e1df17690c6-kube-api-access-5qq2p\") pod \"authorino-58cd955675-fb5f2\" (UID: \"dded7869-e070-4b1a-9438-6e1df17690c6\") " pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:45.395156 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.395120 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qq2p\" (UniqueName: \"kubernetes.io/projected/dded7869-e070-4b1a-9438-6e1df17690c6-kube-api-access-5qq2p\") pod \"authorino-58cd955675-fb5f2\" (UID: \"dded7869-e070-4b1a-9438-6e1df17690c6\") " pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:45.406054 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.406025 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qq2p\" (UniqueName: \"kubernetes.io/projected/dded7869-e070-4b1a-9438-6e1df17690c6-kube-api-access-5qq2p\") pod \"authorino-58cd955675-fb5f2\" (UID: \"dded7869-e070-4b1a-9438-6e1df17690c6\") " pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:45.424158 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.424128 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-vmhv7"] Jul 22 11:57:45.427699 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.427684 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:57:45.437344 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.437323 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vmhv7"] Jul 22 11:57:45.495973 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.495888 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6h7j\" (UniqueName: \"kubernetes.io/projected/0a7ab597-1ccc-469e-b605-c18f5ec9c45e-kube-api-access-f6h7j\") pod \"authorino-75d556bd77-vmhv7\" (UID: \"0a7ab597-1ccc-469e-b605-c18f5ec9c45e\") " pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:57:45.529926 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.529885 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:45.597305 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.597270 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6h7j\" (UniqueName: \"kubernetes.io/projected/0a7ab597-1ccc-469e-b605-c18f5ec9c45e-kube-api-access-f6h7j\") pod \"authorino-75d556bd77-vmhv7\" (UID: \"0a7ab597-1ccc-469e-b605-c18f5ec9c45e\") " pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:57:45.606739 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.606718 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6h7j\" (UniqueName: \"kubernetes.io/projected/0a7ab597-1ccc-469e-b605-c18f5ec9c45e-kube-api-access-f6h7j\") pod \"authorino-75d556bd77-vmhv7\" (UID: \"0a7ab597-1ccc-469e-b605-c18f5ec9c45e\") " pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:57:45.669102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.669077 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-fb5f2"] Jul 22 11:57:45.670748 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:57:45.670722 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddded7869_e070_4b1a_9438_6e1df17690c6.slice/crio-1062c74f3a6334a248aac43b4357d2ac3d46ed64376f194bfc4fdf98fe7668b9 WatchSource:0}: Error finding container 1062c74f3a6334a248aac43b4357d2ac3d46ed64376f194bfc4fdf98fe7668b9: Status 404 returned error can't find the container with id 1062c74f3a6334a248aac43b4357d2ac3d46ed64376f194bfc4fdf98fe7668b9 Jul 22 11:57:45.714761 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.714724 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fb5f2" event={"ID":"dded7869-e070-4b1a-9438-6e1df17690c6","Type":"ContainerStarted","Data":"1062c74f3a6334a248aac43b4357d2ac3d46ed64376f194bfc4fdf98fe7668b9"} Jul 22 11:57:45.736889 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.736862 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:57:45.876113 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:45.876086 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vmhv7"] Jul 22 11:57:45.878345 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:57:45.878314 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a7ab597_1ccc_469e_b605_c18f5ec9c45e.slice/crio-0767a1fe5e2fd29e948f439b501ec53c76c0bd263755b867b9a578b1e586f245 WatchSource:0}: Error finding container 0767a1fe5e2fd29e948f439b501ec53c76c0bd263755b867b9a578b1e586f245: Status 404 returned error can't find the container with id 0767a1fe5e2fd29e948f439b501ec53c76c0bd263755b867b9a578b1e586f245 Jul 22 11:57:46.728262 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:46.728202 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vmhv7" event={"ID":"0a7ab597-1ccc-469e-b605-c18f5ec9c45e","Type":"ContainerStarted","Data":"0767a1fe5e2fd29e948f439b501ec53c76c0bd263755b867b9a578b1e586f245"} Jul 22 11:57:49.747986 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:49.747947 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vmhv7" event={"ID":"0a7ab597-1ccc-469e-b605-c18f5ec9c45e","Type":"ContainerStarted","Data":"15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45"} Jul 22 11:57:49.749413 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:49.749387 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fb5f2" event={"ID":"dded7869-e070-4b1a-9438-6e1df17690c6","Type":"ContainerStarted","Data":"b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4"} Jul 22 11:57:49.767577 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:49.767526 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-vmhv7" podStartSLOduration=1.351195799 podStartE2EDuration="4.767510852s" totalImagesPullingTime="3.416315053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:49.766460601 +0000 UTC m=+498.092307867" watchObservedRunningTime="2026-07-22 11:57:49.767510852 +0000 UTC m=+498.093358116" Jul 22 11:57:49.785942 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:49.785880 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-fb5f2" podStartSLOduration=1.173106218 podStartE2EDuration="4.785858919s" totalImagesPullingTime="3.612752701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:49.784600311 +0000 UTC m=+498.110447577" watchObservedRunningTime="2026-07-22 11:57:49.785858919 +0000 UTC m=+498.111706186" Jul 22 11:57:49.824422 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:49.824387 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-fb5f2"] Jul 22 11:57:51.759820 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:51.759759 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-fb5f2" podUID="dded7869-e070-4b1a-9438-6e1df17690c6" containerName="authorino" containerID="cri-o://b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4" gracePeriod=30 Jul 22 11:57:52.009896 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.009841 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:52.162857 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.162827 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qq2p\" (UniqueName: \"kubernetes.io/projected/dded7869-e070-4b1a-9438-6e1df17690c6-kube-api-access-5qq2p\") pod \"dded7869-e070-4b1a-9438-6e1df17690c6\" (UID: \"dded7869-e070-4b1a-9438-6e1df17690c6\") " Jul 22 11:57:52.164933 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.164913 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dded7869-e070-4b1a-9438-6e1df17690c6-kube-api-access-5qq2p" (OuterVolumeSpecName: "kube-api-access-5qq2p") pod "dded7869-e070-4b1a-9438-6e1df17690c6" (UID: "dded7869-e070-4b1a-9438-6e1df17690c6"). InnerVolumeSpecName "kube-api-access-5qq2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:57:52.263818 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.263731 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qq2p\" (UniqueName: \"kubernetes.io/projected/dded7869-e070-4b1a-9438-6e1df17690c6-kube-api-access-5qq2p\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:57:52.766399 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.766363 2534 generic.go:356] "Generic (PLEG): container finished" podID="dded7869-e070-4b1a-9438-6e1df17690c6" containerID="b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4" exitCode=0 Jul 22 11:57:52.766830 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.766408 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fb5f2" Jul 22 11:57:52.766830 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.766409 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fb5f2" event={"ID":"dded7869-e070-4b1a-9438-6e1df17690c6","Type":"ContainerDied","Data":"b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4"} Jul 22 11:57:52.766830 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.766449 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fb5f2" event={"ID":"dded7869-e070-4b1a-9438-6e1df17690c6","Type":"ContainerDied","Data":"1062c74f3a6334a248aac43b4357d2ac3d46ed64376f194bfc4fdf98fe7668b9"} Jul 22 11:57:52.766830 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.766467 2534 scope.go:117] "RemoveContainer" containerID="b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4" Jul 22 11:57:52.776039 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.776020 2534 scope.go:117] "RemoveContainer" containerID="b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4" Jul 22 11:57:52.776263 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:57:52.776242 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4\": container with ID starting with b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4 not found: ID does not exist" containerID="b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4" Jul 22 11:57:52.776312 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.776270 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4"} err="failed to get container status \"b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4\": rpc error: code = NotFound desc = could not find container \"b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4\": container with ID starting with b2ab044968220ad974070a3f44a715c03c8015389badb6109d6b77a22af397d4 not found: ID does not exist" Jul 22 11:57:52.790760 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.790739 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-fb5f2"] Jul 22 11:57:52.796007 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:52.795988 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-fb5f2"] Jul 22 11:57:54.101567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:57:54.101534 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dded7869-e070-4b1a-9438-6e1df17690c6" path="/var/lib/kubelet/pods/dded7869-e070-4b1a-9438-6e1df17690c6/volumes" Jul 22 11:58:01.786305 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:01.786273 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-567d94866c-9c28x"] Jul 22 11:58:18.437081 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.437044 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g5nmr"] Jul 22 11:58:18.437533 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.437402 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dded7869-e070-4b1a-9438-6e1df17690c6" containerName="authorino" Jul 22 11:58:18.437533 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.437414 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="dded7869-e070-4b1a-9438-6e1df17690c6" containerName="authorino" Jul 22 11:58:18.437533 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.437477 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="dded7869-e070-4b1a-9438-6e1df17690c6" containerName="authorino" Jul 22 11:58:18.443911 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.443877 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:18.462388 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.462359 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g5nmr"] Jul 22 11:58:18.489501 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.489472 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pdhh\" (UniqueName: \"kubernetes.io/projected/35ddc16d-3d49-472c-98fc-60014edd6d46-kube-api-access-6pdhh\") pod \"authorino-5d4b67c446-g5nmr\" (UID: \"35ddc16d-3d49-472c-98fc-60014edd6d46\") " pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:18.590880 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.590848 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pdhh\" (UniqueName: \"kubernetes.io/projected/35ddc16d-3d49-472c-98fc-60014edd6d46-kube-api-access-6pdhh\") pod \"authorino-5d4b67c446-g5nmr\" (UID: \"35ddc16d-3d49-472c-98fc-60014edd6d46\") " pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:18.612981 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.612956 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pdhh\" (UniqueName: \"kubernetes.io/projected/35ddc16d-3d49-472c-98fc-60014edd6d46-kube-api-access-6pdhh\") pod \"authorino-5d4b67c446-g5nmr\" (UID: \"35ddc16d-3d49-472c-98fc-60014edd6d46\") " pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:18.711740 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.711638 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g5nmr"] Jul 22 11:58:18.711956 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.711931 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:18.769595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.769569 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5465fd76f6-jq785"] Jul 22 11:58:18.776003 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.775299 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:18.793654 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.793617 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzr7h\" (UniqueName: \"kubernetes.io/projected/9c5231ab-50e9-418f-8cb4-f2960bef2441-kube-api-access-nzr7h\") pod \"authorino-5465fd76f6-jq785\" (UID: \"9c5231ab-50e9-418f-8cb4-f2960bef2441\") " pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:18.799062 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.799030 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5465fd76f6-jq785"] Jul 22 11:58:18.881464 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.881441 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g5nmr"] Jul 22 11:58:18.883552 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:58:18.883530 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35ddc16d_3d49_472c_98fc_60014edd6d46.slice/crio-ad75900a28b1351d95ad845ed8d1f2533172275acb08d42cee2361cac9f4a95b WatchSource:0}: Error finding container ad75900a28b1351d95ad845ed8d1f2533172275acb08d42cee2361cac9f4a95b: Status 404 returned error can't find the container with id ad75900a28b1351d95ad845ed8d1f2533172275acb08d42cee2361cac9f4a95b Jul 22 11:58:18.895126 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.895100 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzr7h\" (UniqueName: \"kubernetes.io/projected/9c5231ab-50e9-418f-8cb4-f2960bef2441-kube-api-access-nzr7h\") pod \"authorino-5465fd76f6-jq785\" (UID: \"9c5231ab-50e9-418f-8cb4-f2960bef2441\") " pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:18.906236 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.906205 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" event={"ID":"35ddc16d-3d49-472c-98fc-60014edd6d46","Type":"ContainerStarted","Data":"ad75900a28b1351d95ad845ed8d1f2533172275acb08d42cee2361cac9f4a95b"} Jul 22 11:58:18.923858 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:18.923830 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzr7h\" (UniqueName: \"kubernetes.io/projected/9c5231ab-50e9-418f-8cb4-f2960bef2441-kube-api-access-nzr7h\") pod \"authorino-5465fd76f6-jq785\" (UID: \"9c5231ab-50e9-418f-8cb4-f2960bef2441\") " pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:19.087314 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.087235 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:19.102916 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.102888 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5465fd76f6-jq785"] Jul 22 11:58:19.165851 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.165822 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-595d4859b-k5xlp"] Jul 22 11:58:19.170261 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.170242 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.174452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.174419 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 11:58:19.197844 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.197812 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tkdl\" (UniqueName: \"kubernetes.io/projected/edc1cb39-3717-4e16-9cff-91c5b8c81845-kube-api-access-9tkdl\") pod \"authorino-595d4859b-k5xlp\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.197994 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.197876 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/edc1cb39-3717-4e16-9cff-91c5b8c81845-tls-cert\") pod \"authorino-595d4859b-k5xlp\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.209893 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.209867 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595d4859b-k5xlp"] Jul 22 11:58:19.260559 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.260517 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5465fd76f6-jq785"] Jul 22 11:58:19.262772 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:58:19.262747 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c5231ab_50e9_418f_8cb4_f2960bef2441.slice/crio-f17bb663f41810e9fbfb3801bd234a25971e9280dd3114769d6966ad21fffb41 WatchSource:0}: Error finding container f17bb663f41810e9fbfb3801bd234a25971e9280dd3114769d6966ad21fffb41: Status 404 returned error can't find the container with id f17bb663f41810e9fbfb3801bd234a25971e9280dd3114769d6966ad21fffb41 Jul 22 11:58:19.299067 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.299037 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tkdl\" (UniqueName: \"kubernetes.io/projected/edc1cb39-3717-4e16-9cff-91c5b8c81845-kube-api-access-9tkdl\") pod \"authorino-595d4859b-k5xlp\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.299251 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.299090 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/edc1cb39-3717-4e16-9cff-91c5b8c81845-tls-cert\") pod \"authorino-595d4859b-k5xlp\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.301606 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.301589 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/edc1cb39-3717-4e16-9cff-91c5b8c81845-tls-cert\") pod \"authorino-595d4859b-k5xlp\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.317111 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.317088 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tkdl\" (UniqueName: \"kubernetes.io/projected/edc1cb39-3717-4e16-9cff-91c5b8c81845-kube-api-access-9tkdl\") pod \"authorino-595d4859b-k5xlp\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.482980 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.482957 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 11:58:19.657441 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.657414 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595d4859b-k5xlp"] Jul 22 11:58:19.658196 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:58:19.658172 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedc1cb39_3717_4e16_9cff_91c5b8c81845.slice/crio-e2f993efc61901bbde30e19d3ac6676ad102ec6f58a350ca367bde67bb3a0177 WatchSource:0}: Error finding container e2f993efc61901bbde30e19d3ac6676ad102ec6f58a350ca367bde67bb3a0177: Status 404 returned error can't find the container with id e2f993efc61901bbde30e19d3ac6676ad102ec6f58a350ca367bde67bb3a0177 Jul 22 11:58:19.914259 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.914167 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d4859b-k5xlp" event={"ID":"edc1cb39-3717-4e16-9cff-91c5b8c81845","Type":"ContainerStarted","Data":"e2f993efc61901bbde30e19d3ac6676ad102ec6f58a350ca367bde67bb3a0177"} Jul 22 11:58:19.915781 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.915749 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5465fd76f6-jq785" event={"ID":"9c5231ab-50e9-418f-8cb4-f2960bef2441","Type":"ContainerStarted","Data":"e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5"} Jul 22 11:58:19.915939 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.915747 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5465fd76f6-jq785" podUID="9c5231ab-50e9-418f-8cb4-f2960bef2441" containerName="authorino" containerID="cri-o://e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5" gracePeriod=30 Jul 22 11:58:19.915939 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.915818 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5465fd76f6-jq785" event={"ID":"9c5231ab-50e9-418f-8cb4-f2960bef2441","Type":"ContainerStarted","Data":"f17bb663f41810e9fbfb3801bd234a25971e9280dd3114769d6966ad21fffb41"} Jul 22 11:58:19.917533 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.917508 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" event={"ID":"35ddc16d-3d49-472c-98fc-60014edd6d46","Type":"ContainerStarted","Data":"edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3"} Jul 22 11:58:19.917622 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.917587 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" podUID="35ddc16d-3d49-472c-98fc-60014edd6d46" containerName="authorino" containerID="cri-o://edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3" gracePeriod=30 Jul 22 11:58:19.958625 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.958575 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5465fd76f6-jq785" podStartSLOduration=1.462767893 podStartE2EDuration="1.95855841s" totalImagesPullingTime="495.790517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:58:19.958018664 +0000 UTC m=+528.283865931" watchObservedRunningTime="2026-07-22 11:58:19.95855841 +0000 UTC m=+528.284405652" Jul 22 11:58:19.993523 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:19.993467 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" podStartSLOduration=1.472444813 podStartE2EDuration="1.993449594s" totalImagesPullingTime="521.004781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:58:19.991745512 +0000 UTC m=+528.317592776" watchObservedRunningTime="2026-07-22 11:58:19.993449594 +0000 UTC m=+528.319296858" Jul 22 11:58:20.165629 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.165546 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:20.195187 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.195160 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:20.207397 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.207368 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pdhh\" (UniqueName: \"kubernetes.io/projected/35ddc16d-3d49-472c-98fc-60014edd6d46-kube-api-access-6pdhh\") pod \"35ddc16d-3d49-472c-98fc-60014edd6d46\" (UID: \"35ddc16d-3d49-472c-98fc-60014edd6d46\") " Jul 22 11:58:20.207397 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.207396 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzr7h\" (UniqueName: \"kubernetes.io/projected/9c5231ab-50e9-418f-8cb4-f2960bef2441-kube-api-access-nzr7h\") pod \"9c5231ab-50e9-418f-8cb4-f2960bef2441\" (UID: \"9c5231ab-50e9-418f-8cb4-f2960bef2441\") " Jul 22 11:58:20.210537 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.210507 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35ddc16d-3d49-472c-98fc-60014edd6d46-kube-api-access-6pdhh" (OuterVolumeSpecName: "kube-api-access-6pdhh") pod "35ddc16d-3d49-472c-98fc-60014edd6d46" (UID: "35ddc16d-3d49-472c-98fc-60014edd6d46"). InnerVolumeSpecName "kube-api-access-6pdhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:58:20.210662 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.210511 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5231ab-50e9-418f-8cb4-f2960bef2441-kube-api-access-nzr7h" (OuterVolumeSpecName: "kube-api-access-nzr7h") pod "9c5231ab-50e9-418f-8cb4-f2960bef2441" (UID: "9c5231ab-50e9-418f-8cb4-f2960bef2441"). InnerVolumeSpecName "kube-api-access-nzr7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:58:20.308784 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.308751 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pdhh\" (UniqueName: \"kubernetes.io/projected/35ddc16d-3d49-472c-98fc-60014edd6d46-kube-api-access-6pdhh\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:20.308784 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.308778 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzr7h\" (UniqueName: \"kubernetes.io/projected/9c5231ab-50e9-418f-8cb4-f2960bef2441-kube-api-access-nzr7h\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:20.924310 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.924279 2534 generic.go:356] "Generic (PLEG): container finished" podID="35ddc16d-3d49-472c-98fc-60014edd6d46" containerID="edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3" exitCode=0 Jul 22 11:58:20.924777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.924323 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" Jul 22 11:58:20.924777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.924375 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" event={"ID":"35ddc16d-3d49-472c-98fc-60014edd6d46","Type":"ContainerDied","Data":"edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3"} Jul 22 11:58:20.924777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.924416 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g5nmr" event={"ID":"35ddc16d-3d49-472c-98fc-60014edd6d46","Type":"ContainerDied","Data":"ad75900a28b1351d95ad845ed8d1f2533172275acb08d42cee2361cac9f4a95b"} Jul 22 11:58:20.924777 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.924441 2534 scope.go:117] "RemoveContainer" containerID="edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3" Jul 22 11:58:20.926061 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.926020 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d4859b-k5xlp" event={"ID":"edc1cb39-3717-4e16-9cff-91c5b8c81845","Type":"ContainerStarted","Data":"d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4"} Jul 22 11:58:20.927354 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.927332 2534 generic.go:356] "Generic (PLEG): container finished" podID="9c5231ab-50e9-418f-8cb4-f2960bef2441" containerID="e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5" exitCode=0 Jul 22 11:58:20.927452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.927374 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5465fd76f6-jq785" Jul 22 11:58:20.927452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.927421 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5465fd76f6-jq785" event={"ID":"9c5231ab-50e9-418f-8cb4-f2960bef2441","Type":"ContainerDied","Data":"e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5"} Jul 22 11:58:20.927570 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.927458 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5465fd76f6-jq785" event={"ID":"9c5231ab-50e9-418f-8cb4-f2960bef2441","Type":"ContainerDied","Data":"f17bb663f41810e9fbfb3801bd234a25971e9280dd3114769d6966ad21fffb41"} Jul 22 11:58:20.935083 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.935067 2534 scope.go:117] "RemoveContainer" containerID="edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3" Jul 22 11:58:20.935339 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:58:20.935324 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3\": container with ID starting with edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3 not found: ID does not exist" containerID="edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3" Jul 22 11:58:20.935384 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.935348 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3"} err="failed to get container status \"edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3\": rpc error: code = NotFound desc = could not find container \"edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3\": container with ID starting with edc08271068ee2ce7c839da8519f2cce90a976566be0efb4d9fe0358dc7561a3 not found: ID does not exist" Jul 22 11:58:20.935384 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.935364 2534 scope.go:117] "RemoveContainer" containerID="e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5" Jul 22 11:58:20.943251 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.943233 2534 scope.go:117] "RemoveContainer" containerID="e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5" Jul 22 11:58:20.943500 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:58:20.943484 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5\": container with ID starting with e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5 not found: ID does not exist" containerID="e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5" Jul 22 11:58:20.943556 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.943507 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5"} err="failed to get container status \"e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5\": rpc error: code = NotFound desc = could not find container \"e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5\": container with ID starting with e286865790a2214b4a998af76a9551f01d7ca9c5a5b59d373724914d8369a4f5 not found: ID does not exist" Jul 22 11:58:20.964093 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.964051 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-595d4859b-k5xlp" podStartSLOduration=1.584252258 podStartE2EDuration="1.964039303s" totalImagesPullingTime="379.787045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:58:20.963434513 +0000 UTC m=+529.289281777" watchObservedRunningTime="2026-07-22 11:58:20.964039303 +0000 UTC m=+529.289886567" Jul 22 11:58:20.991039 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.991013 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5465fd76f6-jq785"] Jul 22 11:58:20.995448 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:20.995425 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5465fd76f6-jq785"] Jul 22 11:58:21.020779 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.020756 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g5nmr"] Jul 22 11:58:21.029189 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.028175 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vmhv7"] Jul 22 11:58:21.029189 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.028242 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g5nmr"] Jul 22 11:58:21.029189 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.028439 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-vmhv7" podUID="0a7ab597-1ccc-469e-b605-c18f5ec9c45e" containerName="authorino" containerID="cri-o://15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45" gracePeriod=30 Jul 22 11:58:21.284605 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.284578 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:58:21.317839 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.317778 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6h7j\" (UniqueName: \"kubernetes.io/projected/0a7ab597-1ccc-469e-b605-c18f5ec9c45e-kube-api-access-f6h7j\") pod \"0a7ab597-1ccc-469e-b605-c18f5ec9c45e\" (UID: \"0a7ab597-1ccc-469e-b605-c18f5ec9c45e\") " Jul 22 11:58:21.319876 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.319847 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7ab597-1ccc-469e-b605-c18f5ec9c45e-kube-api-access-f6h7j" (OuterVolumeSpecName: "kube-api-access-f6h7j") pod "0a7ab597-1ccc-469e-b605-c18f5ec9c45e" (UID: "0a7ab597-1ccc-469e-b605-c18f5ec9c45e"). InnerVolumeSpecName "kube-api-access-f6h7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:58:21.418872 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.418830 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6h7j\" (UniqueName: \"kubernetes.io/projected/0a7ab597-1ccc-469e-b605-c18f5ec9c45e-kube-api-access-f6h7j\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:21.935509 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.935477 2534 generic.go:356] "Generic (PLEG): container finished" podID="0a7ab597-1ccc-469e-b605-c18f5ec9c45e" containerID="15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45" exitCode=0 Jul 22 11:58:21.935958 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.935519 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vmhv7" Jul 22 11:58:21.935958 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.935546 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vmhv7" event={"ID":"0a7ab597-1ccc-469e-b605-c18f5ec9c45e","Type":"ContainerDied","Data":"15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45"} Jul 22 11:58:21.935958 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.935582 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vmhv7" event={"ID":"0a7ab597-1ccc-469e-b605-c18f5ec9c45e","Type":"ContainerDied","Data":"0767a1fe5e2fd29e948f439b501ec53c76c0bd263755b867b9a578b1e586f245"} Jul 22 11:58:21.935958 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.935602 2534 scope.go:117] "RemoveContainer" containerID="15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45" Jul 22 11:58:21.944768 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.944749 2534 scope.go:117] "RemoveContainer" containerID="15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45" Jul 22 11:58:21.945058 ip-10-0-143-104 kubenswrapper[2534]: E0722 11:58:21.945037 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45\": container with ID starting with 15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45 not found: ID does not exist" containerID="15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45" Jul 22 11:58:21.945125 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.945066 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45"} err="failed to get container status \"15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45\": rpc error: code = NotFound desc = could not find container \"15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45\": container with ID starting with 15acd9bbd5d277a7563eda3fd08170464ad0b676553e377872fa259c4fea2e45 not found: ID does not exist" Jul 22 11:58:21.962595 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.962563 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vmhv7"] Jul 22 11:58:21.967906 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:21.967883 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vmhv7"] Jul 22 11:58:22.102369 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:22.102337 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7ab597-1ccc-469e-b605-c18f5ec9c45e" path="/var/lib/kubelet/pods/0a7ab597-1ccc-469e-b605-c18f5ec9c45e/volumes" Jul 22 11:58:22.102650 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:22.102638 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35ddc16d-3d49-472c-98fc-60014edd6d46" path="/var/lib/kubelet/pods/35ddc16d-3d49-472c-98fc-60014edd6d46/volumes" Jul 22 11:58:22.102945 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:22.102932 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c5231ab-50e9-418f-8cb4-f2960bef2441" path="/var/lib/kubelet/pods/9c5231ab-50e9-418f-8cb4-f2960bef2441/volumes" Jul 22 11:58:26.808744 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:26.808640 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-567d94866c-9c28x" podUID="e17dda2e-5c90-4922-b059-960d16a383b4" containerName="console" containerID="cri-o://d941556da35888e0fbd6e51d6d298c2504f0c72fd15cc5aa7e3790b09fbf35e6" gracePeriod=15 Jul 22 11:58:26.968457 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:26.968421 2534 generic.go:356] "Generic (PLEG): container finished" podID="e17dda2e-5c90-4922-b059-960d16a383b4" containerID="d941556da35888e0fbd6e51d6d298c2504f0c72fd15cc5aa7e3790b09fbf35e6" exitCode=0 Jul 22 11:58:26.968612 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:26.968491 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d94866c-9c28x" event={"ID":"e17dda2e-5c90-4922-b059-960d16a383b4","Type":"ContainerDied","Data":"d941556da35888e0fbd6e51d6d298c2504f0c72fd15cc5aa7e3790b09fbf35e6"} Jul 22 11:58:27.054174 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.054153 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:58:27.165310 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165279 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-trusted-ca-bundle\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165316 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-console-config\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165347 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-service-ca\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165371 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-oauth-serving-cert\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165421 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-oauth-config\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165444 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-serving-cert\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165500 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165471 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbdxc\" (UniqueName: \"kubernetes.io/projected/e17dda2e-5c90-4922-b059-960d16a383b4-kube-api-access-dbdxc\") pod \"e17dda2e-5c90-4922-b059-960d16a383b4\" (UID: \"e17dda2e-5c90-4922-b059-960d16a383b4\") " Jul 22 11:58:27.165854 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165709 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-console-config" (OuterVolumeSpecName: "console-config") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:58:27.165917 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165871 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-service-ca" (OuterVolumeSpecName: "service-ca") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:58:27.165917 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.165892 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:58:27.166025 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.166004 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:58:27.167891 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.167867 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:58:27.167891 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.167871 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17dda2e-5c90-4922-b059-960d16a383b4-kube-api-access-dbdxc" (OuterVolumeSpecName: "kube-api-access-dbdxc") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "kube-api-access-dbdxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:58:27.168053 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.167900 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e17dda2e-5c90-4922-b059-960d16a383b4" (UID: "e17dda2e-5c90-4922-b059-960d16a383b4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:58:27.266452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266418 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbdxc\" (UniqueName: \"kubernetes.io/projected/e17dda2e-5c90-4922-b059-960d16a383b4-kube-api-access-dbdxc\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.266452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266446 2534 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-trusted-ca-bundle\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.266452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266455 2534 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-console-config\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.266452 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266463 2534 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-service-ca\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.266712 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266472 2534 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e17dda2e-5c90-4922-b059-960d16a383b4-oauth-serving-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.266712 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266480 2534 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-oauth-config\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.266712 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.266488 2534 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e17dda2e-5c90-4922-b059-960d16a383b4-console-serving-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 11:58:27.975419 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.975386 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d94866c-9c28x" Jul 22 11:58:27.975864 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.975381 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d94866c-9c28x" event={"ID":"e17dda2e-5c90-4922-b059-960d16a383b4","Type":"ContainerDied","Data":"66cc45df40096415586d11ad9bc7f9c39c65a9a5694a529d600941a8894fb867"} Jul 22 11:58:27.975864 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:27.975519 2534 scope.go:117] "RemoveContainer" containerID="d941556da35888e0fbd6e51d6d298c2504f0c72fd15cc5aa7e3790b09fbf35e6" Jul 22 11:58:28.012172 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:28.012140 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-567d94866c-9c28x"] Jul 22 11:58:28.019565 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:28.019537 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-567d94866c-9c28x"] Jul 22 11:58:28.101885 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:58:28.101853 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e17dda2e-5c90-4922-b059-960d16a383b4" path="/var/lib/kubelet/pods/e17dda2e-5c90-4922-b059-960d16a383b4/volumes" Jul 22 11:59:00.272413 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:00.272373 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-b8zxh" podUID="83528e59-ca1b-4ae2-b520-6fd11121829e" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 11:59:00.272896 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:00.272503 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:59:00.273282 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:00.273256 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-b8zxh" podUID="83528e59-ca1b-4ae2-b520-6fd11121829e" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 11:59:11.225102 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:11.225070 2534 generic.go:356] "Generic (PLEG): container finished" podID="74dd6e20-18d2-49c7-ba3f-26ad91a1c510" containerID="d81635a8627ee4f9b257d689de2be5f73071221b78e55b6f5b76662b74274588" exitCode=0 Jul 22 11:59:11.225522 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:11.225141 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" event={"ID":"74dd6e20-18d2-49c7-ba3f-26ad91a1c510","Type":"ContainerDied","Data":"d81635a8627ee4f9b257d689de2be5f73071221b78e55b6f5b76662b74274588"} Jul 22 11:59:11.225567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:11.225530 2534 scope.go:117] "RemoveContainer" containerID="d81635a8627ee4f9b257d689de2be5f73071221b78e55b6f5b76662b74274588" Jul 22 11:59:11.226538 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:11.226513 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:59:11.226596 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:11.226583 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:59:11.226632 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:11.226621 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:59:12.243862 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:12.243817 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" event={"ID":"74dd6e20-18d2-49c7-ba3f-26ad91a1c510","Type":"ContainerStarted","Data":"09310a0b39c58093628955ff3bf6526f20f8c17e3170860490e69213f7adfd06"} Jul 22 11:59:13.219947 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:13.219854 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:59:13.220327 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:13.220303 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9nrnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:59:13.220827 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:13.220510 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" podUID="74dd6e20-18d2-49c7-ba3f-26ad91a1c510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 22 11:59:14.220054 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:14.220009 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9nrnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:59:14.220547 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:14.220095 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" podUID="74dd6e20-18d2-49c7-ba3f-26ad91a1c510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 22 11:59:15.219895 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:15.219852 2534 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9nrnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:59:15.220111 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:15.219934 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" podUID="74dd6e20-18d2-49c7-ba3f-26ad91a1c510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 22 11:59:16.223334 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:16.223305 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:59:16.223827 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:16.223697 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:59:16.224333 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:16.224316 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9nrnq" Jul 22 11:59:19.316541 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.316513 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb"] Jul 22 11:59:19.317082 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317064 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7ab597-1ccc-469e-b605-c18f5ec9c45e" containerName="authorino" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317085 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7ab597-1ccc-469e-b605-c18f5ec9c45e" containerName="authorino" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317103 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35ddc16d-3d49-472c-98fc-60014edd6d46" containerName="authorino" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317112 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="35ddc16d-3d49-472c-98fc-60014edd6d46" containerName="authorino" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317122 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c5231ab-50e9-418f-8cb4-f2960bef2441" containerName="authorino" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317130 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c5231ab-50e9-418f-8cb4-f2960bef2441" containerName="authorino" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317148 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e17dda2e-5c90-4922-b059-960d16a383b4" containerName="console" Jul 22 11:59:19.317168 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317156 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e17dda2e-5c90-4922-b059-960d16a383b4" containerName="console" Jul 22 11:59:19.317499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317259 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e17dda2e-5c90-4922-b059-960d16a383b4" containerName="console" Jul 22 11:59:19.317499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317273 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7ab597-1ccc-469e-b605-c18f5ec9c45e" containerName="authorino" Jul 22 11:59:19.317499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317283 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="35ddc16d-3d49-472c-98fc-60014edd6d46" containerName="authorino" Jul 22 11:59:19.317499 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.317294 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c5231ab-50e9-418f-8cb4-f2960bef2441" containerName="authorino" Jul 22 11:59:19.320918 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.320899 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.325045 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.325024 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 11:59:19.326106 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.326083 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 11:59:19.326256 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.326138 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 11:59:19.326381 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.326364 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rx8m7\"" Jul 22 11:59:19.337557 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.337537 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb"] Jul 22 11:59:19.422134 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.422104 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9tk\" (UniqueName: \"kubernetes.io/projected/13f7a223-a224-43d3-a231-7a210b93e7b1-kube-api-access-nw9tk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.422335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.422148 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.422335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.422186 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13f7a223-a224-43d3-a231-7a210b93e7b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.422335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.422228 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.422335 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.422272 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.422488 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.422337 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523008 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13f7a223-a224-43d3-a231-7a210b93e7b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523040 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523047 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523266 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523067 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523266 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523124 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523266 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523177 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9tk\" (UniqueName: \"kubernetes.io/projected/13f7a223-a224-43d3-a231-7a210b93e7b1-kube-api-access-nw9tk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523266 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523219 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523605 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523575 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523720 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523586 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.523720 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.523624 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.525655 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.525614 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13f7a223-a224-43d3-a231-7a210b93e7b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.525752 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.525707 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13f7a223-a224-43d3-a231-7a210b93e7b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.535516 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.535491 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9tk\" (UniqueName: \"kubernetes.io/projected/13f7a223-a224-43d3-a231-7a210b93e7b1-kube-api-access-nw9tk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb\" (UID: \"13f7a223-a224-43d3-a231-7a210b93e7b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.632482 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.632448 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:19.789378 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:19.789350 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb"] Jul 22 11:59:19.791131 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:59:19.791105 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f7a223_a224_43d3_a231_7a210b93e7b1.slice/crio-c8f12e5b5ece904f4a929e0f909e72af8f568d0cdec8493fbf45b2e2ee51dd13 WatchSource:0}: Error finding container c8f12e5b5ece904f4a929e0f909e72af8f568d0cdec8493fbf45b2e2ee51dd13: Status 404 returned error can't find the container with id c8f12e5b5ece904f4a929e0f909e72af8f568d0cdec8493fbf45b2e2ee51dd13 Jul 22 11:59:20.293309 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:20.293269 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" event={"ID":"13f7a223-a224-43d3-a231-7a210b93e7b1","Type":"ContainerStarted","Data":"c8f12e5b5ece904f4a929e0f909e72af8f568d0cdec8493fbf45b2e2ee51dd13"} Jul 22 11:59:28.763122 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.763088 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72"] Jul 22 11:59:28.766649 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.766633 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.774553 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.774533 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 22 11:59:28.783579 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.783548 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72"] Jul 22 11:59:28.804436 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.804410 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06650cb7-9792-4a57-89e8-42421554f281-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.804567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.804455 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.804567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.804482 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.804567 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.804541 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwrrx\" (UniqueName: \"kubernetes.io/projected/06650cb7-9792-4a57-89e8-42421554f281-kube-api-access-xwrrx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.804703 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.804578 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.804703 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.804599 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.905890 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.905851 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906097 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.905903 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906097 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.905941 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwrrx\" (UniqueName: \"kubernetes.io/projected/06650cb7-9792-4a57-89e8-42421554f281-kube-api-access-xwrrx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906097 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.905973 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906261 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.906170 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906261 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.906236 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06650cb7-9792-4a57-89e8-42421554f281-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906409 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.906390 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906469 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.906409 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.906506 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.906480 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.908372 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.908353 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06650cb7-9792-4a57-89e8-42421554f281-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.908669 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.908653 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06650cb7-9792-4a57-89e8-42421554f281-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:28.919110 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:28.919085 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwrrx\" (UniqueName: \"kubernetes.io/projected/06650cb7-9792-4a57-89e8-42421554f281-kube-api-access-xwrrx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cgb72\" (UID: \"06650cb7-9792-4a57-89e8-42421554f281\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:29.077353 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:29.077260 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:29.221562 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:29.221540 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72"] Jul 22 11:59:29.223665 ip-10-0-143-104 kubenswrapper[2534]: W0722 11:59:29.223634 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06650cb7_9792_4a57_89e8_42421554f281.slice/crio-ba2120caa08598f99e3cad074160824516d8ae824febb8a354908ca4fd635e4b WatchSource:0}: Error finding container ba2120caa08598f99e3cad074160824516d8ae824febb8a354908ca4fd635e4b: Status 404 returned error can't find the container with id ba2120caa08598f99e3cad074160824516d8ae824febb8a354908ca4fd635e4b Jul 22 11:59:29.344389 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:29.344305 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" event={"ID":"06650cb7-9792-4a57-89e8-42421554f281","Type":"ContainerStarted","Data":"ba2120caa08598f99e3cad074160824516d8ae824febb8a354908ca4fd635e4b"} Jul 22 11:59:30.273269 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:30.273235 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b8zxh" Jul 22 11:59:32.069093 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:32.069064 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:59:32.070297 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:32.070271 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 11:59:32.367116 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:32.367034 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" event={"ID":"06650cb7-9792-4a57-89e8-42421554f281","Type":"ContainerStarted","Data":"ff02d76d557ea0fed85998fdbe9ca01c036966b889ea428d49d8ea776b3951a1"} Jul 22 11:59:32.368494 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:32.368470 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" event={"ID":"13f7a223-a224-43d3-a231-7a210b93e7b1","Type":"ContainerStarted","Data":"43756dd738c22940527f0a1f1db9504298a91d6d62a86a3a9c8f4f88464711c8"} Jul 22 11:59:32.393710 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:32.393663 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" podStartSLOduration=2.123745993 podStartE2EDuration="4.39364873s" totalImagesPullingTime="2.269902737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:59:32.392075488 +0000 UTC m=+600.717922751" watchObservedRunningTime="2026-07-22 11:59:32.39364873 +0000 UTC m=+600.719495994" Jul 22 11:59:32.418028 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:32.417979 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" podStartSLOduration=1.890120902 podStartE2EDuration="13.417964905s" totalImagesPullingTime="11.527844003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:59:32.415531519 +0000 UTC m=+600.741378784" watchObservedRunningTime="2026-07-22 11:59:32.417964905 +0000 UTC m=+600.743812169" Jul 22 11:59:39.077597 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.077553 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:39.078107 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.077637 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:39.090162 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.090139 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:39.417649 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.417617 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cgb72" Jul 22 11:59:39.633470 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.633442 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:39.633667 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.633503 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:39.646096 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:39.646071 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 11:59:40.423810 ip-10-0-143-104 kubenswrapper[2534]: I0722 11:59:40.423769 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb" Jul 22 12:00:00.208816 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.208710 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5"] Jul 22 12:00:00.212320 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.212304 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" Jul 22 12:00:00.216295 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.216274 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7kmhv\"" Jul 22 12:00:00.222384 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.222366 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" Jul 22 12:00:00.229177 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.229153 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5"] Jul 22 12:00:00.372718 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.372698 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5"] Jul 22 12:00:00.374866 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:00:00.374838 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492cf9bd_4bdf_4f2b_9c34_93ddd9e6e360.slice/crio-a2443aced2d8853bcb7edcd52695b66797a109bdcce3f34b552ef2f8896d499d WatchSource:0}: Error finding container a2443aced2d8853bcb7edcd52695b66797a109bdcce3f34b552ef2f8896d499d: Status 404 returned error can't find the container with id a2443aced2d8853bcb7edcd52695b66797a109bdcce3f34b552ef2f8896d499d Jul 22 12:00:00.376899 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.376880 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 12:00:00.520700 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:00.520613 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" event={"ID":"492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360","Type":"ContainerStarted","Data":"a2443aced2d8853bcb7edcd52695b66797a109bdcce3f34b552ef2f8896d499d"} Jul 22 12:00:01.874414 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.874384 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c"] Jul 22 12:00:01.879000 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.878982 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:01.909503 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.909473 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 22 12:00:01.921623 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.921575 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c"] Jul 22 12:00:01.998279 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.998241 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:01.998478 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.998308 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:01.998478 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.998341 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:01.998478 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.998358 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:01.998478 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.998385 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktt4\" (UniqueName: \"kubernetes.io/projected/9ea5123a-b713-4493-a810-81138edabfa8-kube-api-access-tktt4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:01.998656 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:01.998498 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ea5123a-b713-4493-a810-81138edabfa8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.099591 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099555 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.099816 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099622 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.099816 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099663 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.099816 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099688 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.099816 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099716 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktt4\" (UniqueName: \"kubernetes.io/projected/9ea5123a-b713-4493-a810-81138edabfa8-kube-api-access-tktt4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.100068 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099977 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.100125 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.099785 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ea5123a-b713-4493-a810-81138edabfa8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.100325 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.100300 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.100443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.100396 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.102700 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.102670 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ea5123a-b713-4493-a810-81138edabfa8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.103086 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.103060 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ea5123a-b713-4493-a810-81138edabfa8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.116238 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.116180 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktt4\" (UniqueName: \"kubernetes.io/projected/9ea5123a-b713-4493-a810-81138edabfa8-kube-api-access-tktt4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-txn8c\" (UID: \"9ea5123a-b713-4493-a810-81138edabfa8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.188852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.188747 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:02.390476 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.390438 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c"] Jul 22 12:00:02.394151 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:00:02.394121 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ea5123a_b713_4493_a810_81138edabfa8.slice/crio-a7980f1cc100d4af31e75683ae9b46ad69c41cb22218e299925bb38a964588d0 WatchSource:0}: Error finding container a7980f1cc100d4af31e75683ae9b46ad69c41cb22218e299925bb38a964588d0: Status 404 returned error can't find the container with id a7980f1cc100d4af31e75683ae9b46ad69c41cb22218e299925bb38a964588d0 Jul 22 12:00:02.532734 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:02.532660 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" event={"ID":"9ea5123a-b713-4493-a810-81138edabfa8","Type":"ContainerStarted","Data":"a7980f1cc100d4af31e75683ae9b46ad69c41cb22218e299925bb38a964588d0"} Jul 22 12:00:04.545933 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:04.545837 2534 generic.go:356] "Generic (PLEG): container finished" podID="492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360" containerID="e70b86fd2b17ddd2d9e03d44c3bd1ed66432e5c589d3bdc332071debaac6ad8f" exitCode=0 Jul 22 12:00:04.545933 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:04.545896 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" event={"ID":"492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360","Type":"ContainerDied","Data":"e70b86fd2b17ddd2d9e03d44c3bd1ed66432e5c589d3bdc332071debaac6ad8f"} Jul 22 12:00:05.690341 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:05.690318 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" Jul 22 12:00:06.558867 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:06.558836 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" Jul 22 12:00:06.558867 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:06.558841 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745360-7p9r5" event={"ID":"492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360","Type":"ContainerDied","Data":"a2443aced2d8853bcb7edcd52695b66797a109bdcce3f34b552ef2f8896d499d"} Jul 22 12:00:06.559115 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:06.558930 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2443aced2d8853bcb7edcd52695b66797a109bdcce3f34b552ef2f8896d499d" Jul 22 12:00:31.703593 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:31.703500 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" event={"ID":"9ea5123a-b713-4493-a810-81138edabfa8","Type":"ContainerStarted","Data":"b43f1bb90f778fc62989d3f9b14c37be2390f5d2ad62db8efb1b4bc18c11ac2b"} Jul 22 12:00:31.741378 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:31.741328 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" podStartSLOduration=1.7620150570000002 podStartE2EDuration="30.741312907s" totalImagesPullingTime="28.97929785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:00:31.730860672 +0000 UTC m=+660.056707936" watchObservedRunningTime="2026-07-22 12:00:31.741312907 +0000 UTC m=+660.067160200" Jul 22 12:00:32.189296 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:32.189260 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:32.189296 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:32.189300 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:32.201902 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:32.201880 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:32.720103 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:32.720075 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-txn8c" Jul 22 12:00:47.699367 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.699332 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c86df85c-snhc7"] Jul 22 12:00:47.699884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.699698 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360" containerName="cleanup" Jul 22 12:00:47.699884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.699709 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360" containerName="cleanup" Jul 22 12:00:47.699884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.699775 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360" containerName="cleanup" Jul 22 12:00:47.707179 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.707157 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:47.726165 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.726140 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c86df85c-snhc7"] Jul 22 12:00:47.811192 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.811159 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ecbc3ef-e051-4e60-9240-61d79341b8f6-tls-cert\") pod \"authorino-6c86df85c-snhc7\" (UID: \"0ecbc3ef-e051-4e60-9240-61d79341b8f6\") " pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:47.811357 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.811208 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sf6s\" (UniqueName: \"kubernetes.io/projected/0ecbc3ef-e051-4e60-9240-61d79341b8f6-kube-api-access-9sf6s\") pod \"authorino-6c86df85c-snhc7\" (UID: \"0ecbc3ef-e051-4e60-9240-61d79341b8f6\") " pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:47.912439 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.912411 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ecbc3ef-e051-4e60-9240-61d79341b8f6-tls-cert\") pod \"authorino-6c86df85c-snhc7\" (UID: \"0ecbc3ef-e051-4e60-9240-61d79341b8f6\") " pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:47.912439 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.912444 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sf6s\" (UniqueName: \"kubernetes.io/projected/0ecbc3ef-e051-4e60-9240-61d79341b8f6-kube-api-access-9sf6s\") pod \"authorino-6c86df85c-snhc7\" (UID: \"0ecbc3ef-e051-4e60-9240-61d79341b8f6\") " pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:47.915033 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.915004 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ecbc3ef-e051-4e60-9240-61d79341b8f6-tls-cert\") pod \"authorino-6c86df85c-snhc7\" (UID: \"0ecbc3ef-e051-4e60-9240-61d79341b8f6\") " pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:47.926271 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:47.926243 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sf6s\" (UniqueName: \"kubernetes.io/projected/0ecbc3ef-e051-4e60-9240-61d79341b8f6-kube-api-access-9sf6s\") pod \"authorino-6c86df85c-snhc7\" (UID: \"0ecbc3ef-e051-4e60-9240-61d79341b8f6\") " pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:48.017284 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:48.017193 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c86df85c-snhc7" Jul 22 12:00:48.170230 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:48.170200 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c86df85c-snhc7"] Jul 22 12:00:48.173396 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:00:48.173367 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ecbc3ef_e051_4e60_9240_61d79341b8f6.slice/crio-a73299e8631ef48813d369b00e0e9ad31b8ab4a555212b14b589554d17c48617 WatchSource:0}: Error finding container a73299e8631ef48813d369b00e0e9ad31b8ab4a555212b14b589554d17c48617: Status 404 returned error can't find the container with id a73299e8631ef48813d369b00e0e9ad31b8ab4a555212b14b589554d17c48617 Jul 22 12:00:48.808978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:48.808943 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c86df85c-snhc7" event={"ID":"0ecbc3ef-e051-4e60-9240-61d79341b8f6","Type":"ContainerStarted","Data":"a73299e8631ef48813d369b00e0e9ad31b8ab4a555212b14b589554d17c48617"} Jul 22 12:00:49.818703 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:49.818656 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c86df85c-snhc7" event={"ID":"0ecbc3ef-e051-4e60-9240-61d79341b8f6","Type":"ContainerStarted","Data":"896ea05507af64507303b7ef9e8682036f4c0ae1a83954432ca52ff8a30f4fc9"} Jul 22 12:00:49.849461 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:49.849406 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c86df85c-snhc7" podStartSLOduration=1.86526378 podStartE2EDuration="2.849388191s" totalImagesPullingTime="984.124411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:00:49.847363829 +0000 UTC m=+678.173211093" watchObservedRunningTime="2026-07-22 12:00:49.849388191 +0000 UTC m=+678.175235484" Jul 22 12:00:49.893622 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:49.893577 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-595d4859b-k5xlp"] Jul 22 12:00:49.893842 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:49.893809 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-595d4859b-k5xlp" podUID="edc1cb39-3717-4e16-9cff-91c5b8c81845" containerName="authorino" containerID="cri-o://d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4" gracePeriod=30 Jul 22 12:00:50.154050 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.154021 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 12:00:50.337355 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.337321 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/edc1cb39-3717-4e16-9cff-91c5b8c81845-tls-cert\") pod \"edc1cb39-3717-4e16-9cff-91c5b8c81845\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " Jul 22 12:00:50.337542 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.337398 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tkdl\" (UniqueName: \"kubernetes.io/projected/edc1cb39-3717-4e16-9cff-91c5b8c81845-kube-api-access-9tkdl\") pod \"edc1cb39-3717-4e16-9cff-91c5b8c81845\" (UID: \"edc1cb39-3717-4e16-9cff-91c5b8c81845\") " Jul 22 12:00:50.339602 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.339568 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc1cb39-3717-4e16-9cff-91c5b8c81845-kube-api-access-9tkdl" (OuterVolumeSpecName: "kube-api-access-9tkdl") pod "edc1cb39-3717-4e16-9cff-91c5b8c81845" (UID: "edc1cb39-3717-4e16-9cff-91c5b8c81845"). InnerVolumeSpecName "kube-api-access-9tkdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:00:50.348970 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.348940 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edc1cb39-3717-4e16-9cff-91c5b8c81845-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "edc1cb39-3717-4e16-9cff-91c5b8c81845" (UID: "edc1cb39-3717-4e16-9cff-91c5b8c81845"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 12:00:50.438332 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.438244 2534 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/edc1cb39-3717-4e16-9cff-91c5b8c81845-tls-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:00:50.438332 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.438283 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tkdl\" (UniqueName: \"kubernetes.io/projected/edc1cb39-3717-4e16-9cff-91c5b8c81845-kube-api-access-9tkdl\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:00:50.825966 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.825875 2534 generic.go:356] "Generic (PLEG): container finished" podID="edc1cb39-3717-4e16-9cff-91c5b8c81845" containerID="d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4" exitCode=0 Jul 22 12:00:50.825966 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.825952 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d4859b-k5xlp" Jul 22 12:00:50.826496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.825968 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d4859b-k5xlp" event={"ID":"edc1cb39-3717-4e16-9cff-91c5b8c81845","Type":"ContainerDied","Data":"d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4"} Jul 22 12:00:50.826496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.826012 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d4859b-k5xlp" event={"ID":"edc1cb39-3717-4e16-9cff-91c5b8c81845","Type":"ContainerDied","Data":"e2f993efc61901bbde30e19d3ac6676ad102ec6f58a350ca367bde67bb3a0177"} Jul 22 12:00:50.826496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.826034 2534 scope.go:117] "RemoveContainer" containerID="d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4" Jul 22 12:00:50.836158 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.836134 2534 scope.go:117] "RemoveContainer" containerID="d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4" Jul 22 12:00:50.836473 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:00:50.836454 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4\": container with ID starting with d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4 not found: ID does not exist" containerID="d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4" Jul 22 12:00:50.836562 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.836487 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4"} err="failed to get container status \"d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4\": rpc error: code = NotFound desc = could not find container \"d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4\": container with ID starting with d0e46babf8aaf55ef75763ead6f19d25d6c718360c8d27112dbb70392b5c4ff4 not found: ID does not exist" Jul 22 12:00:50.872396 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.872352 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-595d4859b-k5xlp"] Jul 22 12:00:50.881114 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:50.881082 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-595d4859b-k5xlp"] Jul 22 12:00:52.102837 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:00:52.102782 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edc1cb39-3717-4e16-9cff-91c5b8c81845" path="/var/lib/kubelet/pods/edc1cb39-3717-4e16-9cff-91c5b8c81845/volumes" Jul 22 12:04:32.115213 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:04:32.115137 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:04:32.115980 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:04:32.115957 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:07:58.673376 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:07:58.673299 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz"] Jul 22 12:07:58.673896 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:07:58.673499 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" podUID="e46a5732-7a73-4da7-97a7-e907a87264b6" containerName="manager" containerID="cri-o://475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67" gracePeriod=10 Jul 22 12:08:01.429807 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.429773 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 12:08:01.509695 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.509607 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e46a5732-7a73-4da7-97a7-e907a87264b6-extensions-socket-volume\") pod \"e46a5732-7a73-4da7-97a7-e907a87264b6\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " Jul 22 12:08:01.509695 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.509643 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fff6\" (UniqueName: \"kubernetes.io/projected/e46a5732-7a73-4da7-97a7-e907a87264b6-kube-api-access-6fff6\") pod \"e46a5732-7a73-4da7-97a7-e907a87264b6\" (UID: \"e46a5732-7a73-4da7-97a7-e907a87264b6\") " Jul 22 12:08:01.510067 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.510043 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e46a5732-7a73-4da7-97a7-e907a87264b6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e46a5732-7a73-4da7-97a7-e907a87264b6" (UID: "e46a5732-7a73-4da7-97a7-e907a87264b6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:08:01.511952 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.511928 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e46a5732-7a73-4da7-97a7-e907a87264b6-kube-api-access-6fff6" (OuterVolumeSpecName: "kube-api-access-6fff6") pod "e46a5732-7a73-4da7-97a7-e907a87264b6" (UID: "e46a5732-7a73-4da7-97a7-e907a87264b6"). InnerVolumeSpecName "kube-api-access-6fff6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:08:01.611304 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.611265 2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e46a5732-7a73-4da7-97a7-e907a87264b6-extensions-socket-volume\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:08:01.611304 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:01.611297 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fff6\" (UniqueName: \"kubernetes.io/projected/e46a5732-7a73-4da7-97a7-e907a87264b6-kube-api-access-6fff6\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:08:02.257416 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.257378 2534 generic.go:356] "Generic (PLEG): container finished" podID="e46a5732-7a73-4da7-97a7-e907a87264b6" containerID="475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67" exitCode=0 Jul 22 12:08:02.257588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.257426 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" Jul 22 12:08:02.257588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.257475 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" event={"ID":"e46a5732-7a73-4da7-97a7-e907a87264b6","Type":"ContainerDied","Data":"475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67"} Jul 22 12:08:02.257588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.257528 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz" event={"ID":"e46a5732-7a73-4da7-97a7-e907a87264b6","Type":"ContainerDied","Data":"35d91cd33a9b73d95942740230b038eecbc7b42587019ab4bdc0579dc5c4b333"} Jul 22 12:08:02.257588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.257551 2534 scope.go:117] "RemoveContainer" containerID="475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67" Jul 22 12:08:02.267922 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.267900 2534 scope.go:117] "RemoveContainer" containerID="475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67" Jul 22 12:08:02.268195 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:08:02.268179 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67\": container with ID starting with 475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67 not found: ID does not exist" containerID="475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67" Jul 22 12:08:02.268256 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.268205 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67"} err="failed to get container status \"475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67\": rpc error: code = NotFound desc = could not find container \"475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67\": container with ID starting with 475e044558781c27590f995e68ba7a1e75f1bbe216d317e2bc8a7a94f52f4d67 not found: ID does not exist" Jul 22 12:08:02.282510 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.282485 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz"] Jul 22 12:08:02.284684 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:02.284660 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzqtz"] Jul 22 12:08:04.101689 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:04.101655 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e46a5732-7a73-4da7-97a7-e907a87264b6" path="/var/lib/kubelet/pods/e46a5732-7a73-4da7-97a7-e907a87264b6/volumes" Jul 22 12:08:59.801514 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801433 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5"] Jul 22 12:08:59.801978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801841 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e46a5732-7a73-4da7-97a7-e907a87264b6" containerName="manager" Jul 22 12:08:59.801978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801852 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e46a5732-7a73-4da7-97a7-e907a87264b6" containerName="manager" Jul 22 12:08:59.801978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801866 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc1cb39-3717-4e16-9cff-91c5b8c81845" containerName="authorino" Jul 22 12:08:59.801978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801871 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc1cb39-3717-4e16-9cff-91c5b8c81845" containerName="authorino" Jul 22 12:08:59.801978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801927 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e46a5732-7a73-4da7-97a7-e907a87264b6" containerName="manager" Jul 22 12:08:59.801978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.801937 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc1cb39-3717-4e16-9cff-91c5b8c81845" containerName="authorino" Jul 22 12:08:59.804962 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.804945 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:08:59.807462 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.807443 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pg7tg\"" Jul 22 12:08:59.817113 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.817093 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5"] Jul 22 12:08:59.832762 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.832732 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tbjv5\" (UID: \"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:08:59.832925 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.832848 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6ph6\" (UniqueName: \"kubernetes.io/projected/8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb-kube-api-access-n6ph6\") pod \"kuadrant-operator-controller-manager-86878c769f-tbjv5\" (UID: \"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:08:59.933719 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.933678 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6ph6\" (UniqueName: \"kubernetes.io/projected/8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb-kube-api-access-n6ph6\") pod \"kuadrant-operator-controller-manager-86878c769f-tbjv5\" (UID: \"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:08:59.933719 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.933728 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tbjv5\" (UID: \"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:08:59.934132 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.934116 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tbjv5\" (UID: \"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:08:59.943978 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:08:59.943952 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6ph6\" (UniqueName: \"kubernetes.io/projected/8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb-kube-api-access-n6ph6\") pod \"kuadrant-operator-controller-manager-86878c769f-tbjv5\" (UID: \"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:09:00.115950 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:00.115864 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:09:00.282118 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:00.282087 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5"] Jul 22 12:09:00.283413 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:09:00.283394 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bacb1a4_576f_4da2_ad7b_3b7ca836c2eb.slice/crio-fc2801f16de27668183fde5e97c5e4d2edb08d57cc4c065e6bbba52714bc9238 WatchSource:0}: Error finding container fc2801f16de27668183fde5e97c5e4d2edb08d57cc4c065e6bbba52714bc9238: Status 404 returned error can't find the container with id fc2801f16de27668183fde5e97c5e4d2edb08d57cc4c065e6bbba52714bc9238 Jul 22 12:09:00.568152 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:00.568120 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" event={"ID":"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb","Type":"ContainerStarted","Data":"b31cd51a6981b9288ed41898dd2db88c32b69bb671412e5d92a70aff7fce88fe"} Jul 22 12:09:00.568348 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:00.568172 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" event={"ID":"8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb","Type":"ContainerStarted","Data":"fc2801f16de27668183fde5e97c5e4d2edb08d57cc4c065e6bbba52714bc9238"} Jul 22 12:09:00.568348 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:00.568228 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:09:00.590964 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:00.590919 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" podStartSLOduration=1.590905223 podStartE2EDuration="1.590905223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:00.590303577 +0000 UTC m=+1168.916150841" watchObservedRunningTime="2026-07-22 12:09:00.590905223 +0000 UTC m=+1168.916752486" Jul 22 12:09:11.575931 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:11.575901 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tbjv5" Jul 22 12:09:32.150822 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:32.150772 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:09:32.152827 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:09:32.151123 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:14:32.198326 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:14:32.198202 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:14:32.202584 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:14:32.199700 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:16:10.571224 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.571192 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r"] Jul 22 12:16:10.574701 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.574684 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.577038 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.577019 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-83dec4d6-openshift-default-dockercfg-gt96x\"" Jul 22 12:16:10.587043 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.587016 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r"] Jul 22 12:16:10.649785 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.649748 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-token\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650046 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.649852 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istiod-ca-cert\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650046 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.649903 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-podinfo\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650046 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.649941 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-socket\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650046 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.649980 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-data\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650046 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.650013 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-certs\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650341 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.650058 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-credential-socket\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650341 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.650089 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-envoy\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.650341 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.650128 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxp2c\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-kube-api-access-lxp2c\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751155 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751120 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-podinfo\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751358 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751162 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-socket\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751358 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751213 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-data\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751358 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751249 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-certs\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751511 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751383 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-credential-socket\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751511 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751429 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-envoy\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751511 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751481 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxp2c\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-kube-api-access-lxp2c\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751678 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751524 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-token\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751678 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751563 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istiod-ca-cert\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751876 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751702 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-certs\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751876 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751861 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-socket\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751977 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751886 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-credential-socket\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.751977 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.751949 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-data\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.752633 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.752610 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istiod-ca-cert\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.753934 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.753908 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-envoy\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.754055 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.753976 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-podinfo\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.760987 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.760967 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-token\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.761338 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.761317 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxp2c\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-kube-api-access-lxp2c\") pod \"e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:10.885555 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:10.885524 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:11.029703 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.029681 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r"] Jul 22 12:16:11.031458 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:16:11.031433 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0ec6af_de7e_4192_834d_ac8d5ee7aaca.slice/crio-3324bce83455b1b449d25815cc90ed39b56ee84203b163966eb41f47617f2f88 WatchSource:0}: Error finding container 3324bce83455b1b449d25815cc90ed39b56ee84203b163966eb41f47617f2f88: Status 404 returned error can't find the container with id 3324bce83455b1b449d25815cc90ed39b56ee84203b163966eb41f47617f2f88 Jul 22 12:16:11.033982 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.033950 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:11.034059 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.034017 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:11.034059 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.034042 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:11.962708 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.962671 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" event={"ID":"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca","Type":"ContainerStarted","Data":"b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7"} Jul 22 12:16:11.963208 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.962720 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" event={"ID":"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca","Type":"ContainerStarted","Data":"3324bce83455b1b449d25815cc90ed39b56ee84203b163966eb41f47617f2f88"} Jul 22 12:16:11.984458 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:11.984408 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" podStartSLOduration=1.9843929820000001 podStartE2EDuration="1.984392982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:16:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:16:11.983202415 +0000 UTC m=+1600.309049676" watchObservedRunningTime="2026-07-22 12:16:11.984392982 +0000 UTC m=+1600.310240246" Jul 22 12:16:12.886007 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:12.885974 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:12.890576 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:12.890551 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:12.968320 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:12.968287 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:12.969313 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:12.969293 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:27.053073 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.053037 2534 generic.go:356] "Generic (PLEG): container finished" podID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerID="b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7" exitCode=0 Jul 22 12:16:27.053597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.053099 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" event={"ID":"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca","Type":"ContainerDied","Data":"b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7"} Jul 22 12:16:27.053669 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.053598 2534 scope.go:117] "RemoveContainer" containerID="b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7" Jul 22 12:16:27.054659 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.054630 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:27.054720 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.054701 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:27.054756 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.054732 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:27.969227 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:27.969191 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:28.060886 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:28.060852 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" event={"ID":"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca","Type":"ContainerStarted","Data":"20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837"} Jul 22 12:16:28.885953 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:28.885918 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:28.887136 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:28.887113 2534 patch_prober.go:28] interesting pod/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 12:16:28.887257 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:28.887170 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 12:16:29.886591 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:29.886554 2534 patch_prober.go:28] interesting pod/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 12:16:29.887015 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:29.886609 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 12:16:30.886187 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:30.886153 2534 patch_prober.go:28] interesting pod/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 12:16:30.886368 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:30.886208 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 12:16:31.889619 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:31.889582 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:31.890030 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:31.889830 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:31.890471 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:31.890450 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:54.201267 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:54.201231 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r"] Jul 22 12:16:54.202141 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:54.202087 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" containerID="cri-o://20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837" gracePeriod=30 Jul 22 12:16:55.214784 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.214751 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472"] Jul 22 12:16:55.219081 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.219059 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.221632 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.221615 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-82f25c67-gw-openshift-default-dockercfg-42d5r\"" Jul 22 12:16:55.240130 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.240100 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472"] Jul 22 12:16:55.271155 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271123 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-socket\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271155 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271158 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-data\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271185 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-certs\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271251 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddee7931-b1d8-485a-afb9-ce22a16ab396-istiod-ca-cert\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271313 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-credential-socket\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271340 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-envoy\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271364 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz7kg\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-kube-api-access-rz7kg\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271562 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271383 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-token\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.271562 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.271472 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-podinfo\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372361 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372326 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-certs\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372556 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372367 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddee7931-b1d8-485a-afb9-ce22a16ab396-istiod-ca-cert\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372556 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372425 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-credential-socket\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372556 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372442 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-envoy\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372556 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372468 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz7kg\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-kube-api-access-rz7kg\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372556 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372498 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-token\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372874 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372566 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-podinfo\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372874 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372621 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-socket\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372874 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372652 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-data\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.372874 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372766 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-certs\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.373098 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.372878 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-credential-socket\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.373205 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.373185 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddee7931-b1d8-485a-afb9-ce22a16ab396-istiod-ca-cert\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.373336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.373317 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-socket\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.373569 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.373325 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-data\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.375088 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.375065 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-envoy\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.375228 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.375210 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-podinfo\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.383650 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.383629 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz7kg\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-kube-api-access-rz7kg\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.384043 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.384019 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-token\") pod \"e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.530035 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.529948 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:55.682119 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.682093 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472"] Jul 22 12:16:55.683853 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:16:55.683829 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddee7931_b1d8_485a_afb9_ce22a16ab396.slice/crio-ffa0fd106cb66655f9344275ccc123746854d2135d646c76ba0312c401f612db WatchSource:0}: Error finding container ffa0fd106cb66655f9344275ccc123746854d2135d646c76ba0312c401f612db: Status 404 returned error can't find the container with id ffa0fd106cb66655f9344275ccc123746854d2135d646c76ba0312c401f612db Jul 22 12:16:55.686050 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.686017 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:55.686129 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.686100 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:55.686166 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:55.686131 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:16:56.224812 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:56.224762 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" event={"ID":"ddee7931-b1d8-485a-afb9-ce22a16ab396","Type":"ContainerStarted","Data":"e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895"} Jul 22 12:16:56.225197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:56.224833 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" event={"ID":"ddee7931-b1d8-485a-afb9-ce22a16ab396","Type":"ContainerStarted","Data":"ffa0fd106cb66655f9344275ccc123746854d2135d646c76ba0312c401f612db"} Jul 22 12:16:56.247342 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:56.247292 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" podStartSLOduration=1.247278007 podStartE2EDuration="1.247278007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:16:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:16:56.245731969 +0000 UTC m=+1644.571579244" watchObservedRunningTime="2026-07-22 12:16:56.247278007 +0000 UTC m=+1644.573125273" Jul 22 12:16:56.530957 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:56.530869 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:56.536475 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:56.536450 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:57.230003 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:57.229975 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:57.230861 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:57.230842 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:16:59.460864 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.460835 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:16:59.513774 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.513691 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxp2c\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-kube-api-access-lxp2c\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.513960 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.513827 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-credential-socket\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.513960 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.513876 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-podinfo\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.513960 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.513904 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istiod-ca-cert\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.513960 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.513939 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-socket\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.514193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.513961 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-envoy\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.514193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514003 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-certs\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.514193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514037 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-data\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.514193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514074 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-token\") pod \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\" (UID: \"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca\") " Jul 22 12:16:59.514193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514137 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:16:59.514454 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514300 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:16:59.514454 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514333 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:16:59.514454 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514438 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.514615 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514454 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.514615 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.514470 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.515418 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.515395 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:16:59.515954 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.515928 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:16:59.517208 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.517180 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-kube-api-access-lxp2c" (OuterVolumeSpecName: "kube-api-access-lxp2c") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "kube-api-access-lxp2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:16:59.517343 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.517323 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:16:59.517412 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.517342 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:16:59.517688 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.517671 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" (UID: "bb0ec6af-de7e-4192-834d-ac8d5ee7aaca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:16:59.615588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.615556 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.615588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.615584 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxp2c\" (UniqueName: \"kubernetes.io/projected/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-kube-api-access-lxp2c\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.615588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.615594 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.615788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.615603 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.615788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.615611 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:16:59.615788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:16:59.615618 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:00.248592 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.248555 2534 generic.go:356] "Generic (PLEG): container finished" podID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerID="20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837" exitCode=0 Jul 22 12:17:00.248814 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.248608 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" Jul 22 12:17:00.248814 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.248650 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" event={"ID":"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca","Type":"ContainerDied","Data":"20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837"} Jul 22 12:17:00.248814 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.248703 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r" event={"ID":"bb0ec6af-de7e-4192-834d-ac8d5ee7aaca","Type":"ContainerDied","Data":"3324bce83455b1b449d25815cc90ed39b56ee84203b163966eb41f47617f2f88"} Jul 22 12:17:00.248814 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.248727 2534 scope.go:117] "RemoveContainer" containerID="20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837" Jul 22 12:17:00.257921 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.257902 2534 scope.go:117] "RemoveContainer" containerID="b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7" Jul 22 12:17:00.266379 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.266363 2534 scope.go:117] "RemoveContainer" containerID="20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837" Jul 22 12:17:00.266641 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:17:00.266621 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837\": container with ID starting with 20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837 not found: ID does not exist" containerID="20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837" Jul 22 12:17:00.266705 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.266652 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837"} err="failed to get container status \"20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837\": rpc error: code = NotFound desc = could not find container \"20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837\": container with ID starting with 20c2f5ef78abfd9b891d997d627444893f40fcc61109de8fa02fc3edb468a837 not found: ID does not exist" Jul 22 12:17:00.266705 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.266673 2534 scope.go:117] "RemoveContainer" containerID="b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7" Jul 22 12:17:00.266954 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:17:00.266935 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7\": container with ID starting with b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7 not found: ID does not exist" containerID="b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7" Jul 22 12:17:00.267021 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.266966 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7"} err="failed to get container status \"b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7\": rpc error: code = NotFound desc = could not find container \"b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7\": container with ID starting with b5693e86f86f8cd3c50327826c9e797f78ff6970e1a6ffeda61378edef0563d7 not found: ID does not exist" Jul 22 12:17:00.271554 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.271535 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r"] Jul 22 12:17:00.275830 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:00.275787 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-83dec4d6-openshift-default-6df56bf869-hdn6r"] Jul 22 12:17:02.108597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:02.108566 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" path="/var/lib/kubelet/pods/bb0ec6af-de7e-4192-834d-ac8d5ee7aaca/volumes" Jul 22 12:17:11.311330 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:11.311231 2534 generic.go:356] "Generic (PLEG): container finished" podID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerID="e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895" exitCode=0 Jul 22 12:17:11.311330 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:11.311302 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" event={"ID":"ddee7931-b1d8-485a-afb9-ce22a16ab396","Type":"ContainerDied","Data":"e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895"} Jul 22 12:17:11.311896 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:11.311824 2534 scope.go:117] "RemoveContainer" containerID="e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895" Jul 22 12:17:11.312963 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:11.312932 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:11.313080 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:11.313004 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:11.313080 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:11.313060 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:12.230408 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:12.230364 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:12.322620 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:12.322578 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" event={"ID":"ddee7931-b1d8-485a-afb9-ce22a16ab396","Type":"ContainerStarted","Data":"92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8"} Jul 22 12:17:12.532071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:12.531976 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:12.532767 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:12.532410 2534 patch_prober.go:28] interesting pod/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 12:17:12.532767 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:12.532472 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 12:17:13.531167 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:13.530707 2534 patch_prober.go:28] interesting pod/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 12:17:13.531167 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:13.530781 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 12:17:14.531080 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:14.531040 2534 patch_prober.go:28] interesting pod/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 12:17:14.531362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:14.531331 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 12:17:15.530998 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:15.530954 2534 patch_prober.go:28] interesting pod/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 12:17:15.531328 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:15.531295 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 12:17:16.534574 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:16.534544 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:16.535095 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:16.534830 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:16.535685 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:16.535670 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:38.062010 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.061979 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472"] Jul 22 12:17:38.062500 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.062182 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" containerID="cri-o://92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8" gracePeriod=30 Jul 22 12:17:38.827236 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.827202 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx"] Jul 22 12:17:38.827607 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.827594 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" Jul 22 12:17:38.827672 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.827609 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" Jul 22 12:17:38.827719 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.827685 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" Jul 22 12:17:38.827719 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.827693 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" Jul 22 12:17:38.832709 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.832676 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.835786 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.835644 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-800b6cf7-openshift-default-dockercfg-rbp5d\"" Jul 22 12:17:38.842285 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.842261 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx"] Jul 22 12:17:38.885079 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885052 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-socket\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885088 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-data\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885129 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-envoy\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885148 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-podinfo\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885163 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-token\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885178 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istiod-ca-cert\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885200 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfhsv\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-kube-api-access-sfhsv\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885454 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885261 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-certs\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.885454 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.885281 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-credential-socket\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.986866 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.986836 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-certs\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987033 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.986881 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-credential-socket\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987033 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.986963 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-socket\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987033 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.986996 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-data\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987041 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-envoy\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987065 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-podinfo\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987087 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-token\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987106 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istiod-ca-cert\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987130 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfhsv\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-kube-api-access-sfhsv\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987281 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-certs\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987324 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-socket\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987457 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987368 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-credential-socket\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987768 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987743 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-data\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.987919 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.987904 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istiod-ca-cert\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.989561 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.989541 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-envoy\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.989651 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.989624 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-podinfo\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:38.998390 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:38.998366 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-token\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:39.000680 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.000658 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfhsv\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-kube-api-access-sfhsv\") pod \"e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:39.146183 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.146146 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:39.289535 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.289511 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx"] Jul 22 12:17:39.292290 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:17:39.292257 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71fa403_94cd_4e90_a2d0_e5371133a0c5.slice/crio-ff1d59cf6ef738b778139682dd7b933bfda70ef6e01e0761ecff645dd623ee53 WatchSource:0}: Error finding container ff1d59cf6ef738b778139682dd7b933bfda70ef6e01e0761ecff645dd623ee53: Status 404 returned error can't find the container with id ff1d59cf6ef738b778139682dd7b933bfda70ef6e01e0761ecff645dd623ee53 Jul 22 12:17:39.294315 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.294281 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:39.294408 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.294353 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:39.294408 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.294382 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:39.474472 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.474380 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" event={"ID":"c71fa403-94cd-4e90-a2d0-e5371133a0c5","Type":"ContainerStarted","Data":"9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213"} Jul 22 12:17:39.474472 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.474431 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" event={"ID":"c71fa403-94cd-4e90-a2d0-e5371133a0c5","Type":"ContainerStarted","Data":"ff1d59cf6ef738b778139682dd7b933bfda70ef6e01e0761ecff645dd623ee53"} Jul 22 12:17:39.494984 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:39.494932 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podStartSLOduration=1.494913181 podStartE2EDuration="1.494913181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:17:39.494109935 +0000 UTC m=+1687.819957199" watchObservedRunningTime="2026-07-22 12:17:39.494913181 +0000 UTC m=+1687.820760445" Jul 22 12:17:40.147286 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:40.147255 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:40.152307 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:40.152278 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:40.480193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:40.480104 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:40.480983 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:40.480965 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:43.314154 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.314131 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:43.435754 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.435721 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddee7931-b1d8-485a-afb9-ce22a16ab396-istiod-ca-cert\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.435963 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.435822 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-envoy\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.435963 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.435880 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-socket\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.435963 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.435914 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-data\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.435963 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.435950 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-credential-socket\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.436171 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.435985 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz7kg\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-kube-api-access-rz7kg\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.436171 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436051 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-token\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.436171 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436078 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-certs\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.436171 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436103 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-podinfo\") pod \"ddee7931-b1d8-485a-afb9-ce22a16ab396\" (UID: \"ddee7931-b1d8-485a-afb9-ce22a16ab396\") " Jul 22 12:17:43.436371 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436190 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddee7931-b1d8-485a-afb9-ce22a16ab396-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:17:43.436371 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436308 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:17:43.436371 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436334 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:17:43.436572 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436549 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:17:43.436641 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436599 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddee7931-b1d8-485a-afb9-ce22a16ab396-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.436641 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436621 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.436641 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.436636 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.437213 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.437187 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-data" (OuterVolumeSpecName: "istio-data") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:17:43.438370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.438345 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:17:43.438370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.438354 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-kube-api-access-rz7kg" (OuterVolumeSpecName: "kube-api-access-rz7kg") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "kube-api-access-rz7kg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:17:43.438880 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.438857 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:17:43.438880 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.438862 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-token" (OuterVolumeSpecName: "istio-token") pod "ddee7931-b1d8-485a-afb9-ce22a16ab396" (UID: "ddee7931-b1d8-485a-afb9-ce22a16ab396"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:17:43.497185 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.497155 2534 generic.go:356] "Generic (PLEG): container finished" podID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerID="92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8" exitCode=0 Jul 22 12:17:43.497359 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.497212 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" Jul 22 12:17:43.497359 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.497215 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" event={"ID":"ddee7931-b1d8-485a-afb9-ce22a16ab396","Type":"ContainerDied","Data":"92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8"} Jul 22 12:17:43.497359 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.497283 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472" event={"ID":"ddee7931-b1d8-485a-afb9-ce22a16ab396","Type":"ContainerDied","Data":"ffa0fd106cb66655f9344275ccc123746854d2135d646c76ba0312c401f612db"} Jul 22 12:17:43.497359 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.497304 2534 scope.go:117] "RemoveContainer" containerID="92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8" Jul 22 12:17:43.507880 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.507866 2534 scope.go:117] "RemoveContainer" containerID="e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895" Jul 22 12:17:43.516940 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.516887 2534 scope.go:117] "RemoveContainer" containerID="92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8" Jul 22 12:17:43.517228 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:17:43.517209 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8\": container with ID starting with 92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8 not found: ID does not exist" containerID="92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8" Jul 22 12:17:43.517337 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.517241 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8"} err="failed to get container status \"92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8\": rpc error: code = NotFound desc = could not find container \"92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8\": container with ID starting with 92cbd706fbdd083f991382e063ecc1c139b2f1c6e587e72065b638802f1cb2e8 not found: ID does not exist" Jul 22 12:17:43.517337 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.517265 2534 scope.go:117] "RemoveContainer" containerID="e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895" Jul 22 12:17:43.517578 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:17:43.517563 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895\": container with ID starting with e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895 not found: ID does not exist" containerID="e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895" Jul 22 12:17:43.517634 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.517586 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895"} err="failed to get container status \"e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895\": rpc error: code = NotFound desc = could not find container \"e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895\": container with ID starting with e2f89deaf4f71e14c55fdd0fbbcab715c15ca61368cee754017a836aa55ef895 not found: ID does not exist" Jul 22 12:17:43.522732 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.522708 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472"] Jul 22 12:17:43.525048 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.525029 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-82f25c67-gw-openshift-default-7c89d889cf-f9472"] Jul 22 12:17:43.537113 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.537093 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.537113 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.537112 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.537251 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.537121 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz7kg\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-kube-api-access-rz7kg\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.537251 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.537130 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.537251 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.537140 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddee7931-b1d8-485a-afb9-ce22a16ab396-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:43.537251 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:43.537147 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddee7931-b1d8-485a-afb9-ce22a16ab396-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:17:44.101144 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:44.101114 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" path="/var/lib/kubelet/pods/ddee7931-b1d8-485a-afb9-ce22a16ab396/volumes" Jul 22 12:17:55.481772 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.481738 2534 patch_prober.go:28] interesting pod/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 12:17:55.482191 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.481826 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 12:17:55.573927 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.573894 2534 generic.go:356] "Generic (PLEG): container finished" podID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerID="9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213" exitCode=0 Jul 22 12:17:55.574086 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.573965 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" event={"ID":"c71fa403-94cd-4e90-a2d0-e5371133a0c5","Type":"ContainerDied","Data":"9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213"} Jul 22 12:17:55.574353 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.574338 2534 scope.go:117] "RemoveContainer" containerID="9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213" Jul 22 12:17:55.575370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.575340 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:55.575447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.575428 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:55.575488 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:55.575477 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:17:56.581218 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:56.581178 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" event={"ID":"c71fa403-94cd-4e90-a2d0-e5371133a0c5","Type":"ContainerStarted","Data":"3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078"} Jul 22 12:17:57.147013 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:57.146977 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:17:57.147216 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:57.147197 2534 patch_prober.go:28] interesting pod/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 12:17:57.147266 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:57.147247 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 12:17:58.147176 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:58.147143 2534 patch_prober.go:28] interesting pod/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 12:17:58.147577 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:58.147198 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 12:17:59.147602 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:59.147569 2534 patch_prober.go:28] interesting pod/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 12:17:59.148057 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:17:59.147639 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 12:18:00.146988 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:00.146949 2534 patch_prober.go:28] interesting pod/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 12:18:00.147169 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:00.147040 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 12:18:01.150480 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:01.150448 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:18:01.150958 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:01.150725 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:18:01.151342 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:01.151326 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:18:22.988944 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:22.988900 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx"] Jul 22 12:18:22.989481 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:22.989142 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" containerID="cri-o://3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078" gracePeriod=30 Jul 22 12:18:23.727367 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727332 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47"] Jul 22 12:18:23.727902 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727883 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" Jul 22 12:18:23.727902 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727902 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0ec6af-de7e-4192-834d-ac8d5ee7aaca" containerName="istio-proxy" Jul 22 12:18:23.728052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727916 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" Jul 22 12:18:23.728052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727921 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" Jul 22 12:18:23.728052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727931 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" Jul 22 12:18:23.728052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.727938 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" Jul 22 12:18:23.728052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.728012 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" Jul 22 12:18:23.731189 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.731171 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.733838 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.733816 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-94d621ea-openshift-default-dockercfg-r99h5\"" Jul 22 12:18:23.743911 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.743888 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47"] Jul 22 12:18:23.821636 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821591 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-envoy\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.821862 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821652 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlx6j\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-kube-api-access-nlx6j\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.821862 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821749 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-certs\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.821862 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821784 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-credential-socket\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.821862 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821843 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-socket\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.822110 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821866 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-data\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.822110 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821903 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab14aac7-a10d-4167-9c49-7b5b484b7802-istiod-ca-cert\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.822110 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.821951 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-token\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.822110 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.822016 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-podinfo\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.922870 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.922830 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-certs\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.922870 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.922873 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-credential-socket\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.922891 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-socket\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.922907 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-data\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.922940 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab14aac7-a10d-4167-9c49-7b5b484b7802-istiod-ca-cert\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.922972 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-token\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923008 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-podinfo\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923387 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923228 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-envoy\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923387 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923258 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlx6j\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-kube-api-access-nlx6j\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923387 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923343 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-credential-socket\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923550 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923397 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-socket\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923550 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923452 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-data\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923550 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923507 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-certs\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.923672 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.923654 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab14aac7-a10d-4167-9c49-7b5b484b7802-istiod-ca-cert\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.925555 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.925528 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-envoy\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.925903 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.925881 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-podinfo\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.933100 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.933076 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlx6j\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-kube-api-access-nlx6j\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:23.933835 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:23.933808 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-token\") pod \"e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:24.041395 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.041312 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:24.189632 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.189608 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47"] Jul 22 12:18:24.191448 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:18:24.191422 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab14aac7_a10d_4167_9c49_7b5b484b7802.slice/crio-e36a434db410792a8426a86db47a9317ba1d0380ffcf404bac447862f5a31b44 WatchSource:0}: Error finding container e36a434db410792a8426a86db47a9317ba1d0380ffcf404bac447862f5a31b44: Status 404 returned error can't find the container with id e36a434db410792a8426a86db47a9317ba1d0380ffcf404bac447862f5a31b44 Jul 22 12:18:24.193571 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.193537 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:18:24.193671 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.193599 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:18:24.193671 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.193625 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:18:24.743116 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.743074 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" event={"ID":"ab14aac7-a10d-4167-9c49-7b5b484b7802","Type":"ContainerStarted","Data":"1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a"} Jul 22 12:18:24.743320 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.743134 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" event={"ID":"ab14aac7-a10d-4167-9c49-7b5b484b7802","Type":"ContainerStarted","Data":"e36a434db410792a8426a86db47a9317ba1d0380ffcf404bac447862f5a31b44"} Jul 22 12:18:24.765808 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:24.765729 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" podStartSLOduration=1.765713436 podStartE2EDuration="1.765713436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:18:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:18:24.765291728 +0000 UTC m=+1733.091139005" watchObservedRunningTime="2026-07-22 12:18:24.765713436 +0000 UTC m=+1733.091560700" Jul 22 12:18:25.041784 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:25.041693 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:25.046688 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:25.046661 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:25.749148 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:25.749120 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:25.750009 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:25.749990 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:28.245512 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.245488 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:18:28.365304 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365221 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-podinfo\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365304 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365260 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-credential-socket\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365304 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365294 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-token\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365592 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365315 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-data\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365592 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365342 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istiod-ca-cert\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365592 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365357 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-certs\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365592 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365374 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-envoy\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365592 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365460 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfhsv\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-kube-api-access-sfhsv\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.365875 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365650 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:18:28.365875 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365667 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:18:28.365875 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365764 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:18:28.365875 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.365816 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-socket\") pod \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\" (UID: \"c71fa403-94cd-4e90-a2d0-e5371133a0c5\") " Jul 22 12:18:28.366195 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.366170 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:18:28.366284 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.366261 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.366284 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.366281 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.366561 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.366295 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.366561 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.366309 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.366561 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.366316 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:18:28.367634 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.367612 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:18:28.367763 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.367643 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:18:28.368022 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.368005 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-kube-api-access-sfhsv" (OuterVolumeSpecName: "kube-api-access-sfhsv") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "kube-api-access-sfhsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:18:28.368669 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.368652 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c71fa403-94cd-4e90-a2d0-e5371133a0c5" (UID: "c71fa403-94cd-4e90-a2d0-e5371133a0c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:18:28.467496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.467466 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.467496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.467494 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.467496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.467503 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.467725 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.467512 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfhsv\" (UniqueName: \"kubernetes.io/projected/c71fa403-94cd-4e90-a2d0-e5371133a0c5-kube-api-access-sfhsv\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.467725 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.467521 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c71fa403-94cd-4e90-a2d0-e5371133a0c5-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:18:28.768713 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.768682 2534 generic.go:356] "Generic (PLEG): container finished" podID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerID="3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078" exitCode=0 Jul 22 12:18:28.768935 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.768744 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" Jul 22 12:18:28.768935 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.768776 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" event={"ID":"c71fa403-94cd-4e90-a2d0-e5371133a0c5","Type":"ContainerDied","Data":"3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078"} Jul 22 12:18:28.768935 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.768862 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx" event={"ID":"c71fa403-94cd-4e90-a2d0-e5371133a0c5","Type":"ContainerDied","Data":"ff1d59cf6ef738b778139682dd7b933bfda70ef6e01e0761ecff645dd623ee53"} Jul 22 12:18:28.768935 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.768893 2534 scope.go:117] "RemoveContainer" containerID="3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078" Jul 22 12:18:28.779421 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.779402 2534 scope.go:117] "RemoveContainer" containerID="9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213" Jul 22 12:18:28.787848 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.787824 2534 scope.go:117] "RemoveContainer" containerID="3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078" Jul 22 12:18:28.788102 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:18:28.788082 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078\": container with ID starting with 3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078 not found: ID does not exist" containerID="3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078" Jul 22 12:18:28.788153 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.788112 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078"} err="failed to get container status \"3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078\": rpc error: code = NotFound desc = could not find container \"3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078\": container with ID starting with 3eca8d7469fe436f7c1cc8f43e808d824722ec68187532a54965a36175dbe078 not found: ID does not exist" Jul 22 12:18:28.788153 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.788130 2534 scope.go:117] "RemoveContainer" containerID="9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213" Jul 22 12:18:28.788331 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:18:28.788311 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213\": container with ID starting with 9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213 not found: ID does not exist" containerID="9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213" Jul 22 12:18:28.788378 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.788339 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213"} err="failed to get container status \"9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213\": rpc error: code = NotFound desc = could not find container \"9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213\": container with ID starting with 9129ff7c0eeb6e0cabba27a3cf4237b26ec472bdd402d01486ed15b786ae4213 not found: ID does not exist" Jul 22 12:18:28.793522 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.793497 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx"] Jul 22 12:18:28.798540 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:28.798515 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-800b6cf7-openshift-default-9cbdb5d67-rxkkx"] Jul 22 12:18:30.101775 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:30.101741 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" path="/var/lib/kubelet/pods/c71fa403-94cd-4e90-a2d0-e5371133a0c5/volumes" Jul 22 12:18:36.825264 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:36.825230 2534 generic.go:356] "Generic (PLEG): container finished" podID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerID="1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a" exitCode=0 Jul 22 12:18:36.825689 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:36.825304 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" event={"ID":"ab14aac7-a10d-4167-9c49-7b5b484b7802","Type":"ContainerDied","Data":"1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a"} Jul 22 12:18:36.825746 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:36.825732 2534 scope.go:117] "RemoveContainer" containerID="1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a" Jul 22 12:18:36.826873 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:36.826844 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:18:36.826947 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:36.826925 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:18:36.826992 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:36.826975 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:18:37.838611 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:37.838562 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" event={"ID":"ab14aac7-a10d-4167-9c49-7b5b484b7802","Type":"ContainerStarted","Data":"05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b"} Jul 22 12:18:38.042164 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:38.042131 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:38.043386 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:38.043353 2534 patch_prober.go:28] interesting pod/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 12:18:38.043534 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:38.043404 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 12:18:39.042441 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:39.042403 2534 patch_prober.go:28] interesting pod/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 12:18:39.042863 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:39.042457 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 12:18:40.042053 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:40.042019 2534 patch_prober.go:28] interesting pod/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 12:18:40.042244 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:40.042079 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 12:18:40.749895 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:40.749861 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:41.042247 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:41.042155 2534 patch_prober.go:28] interesting pod/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 22 12:18:41.042247 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:41.042213 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 22 12:18:42.046420 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:42.046387 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:18:42.047370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:18:42.047349 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:19:06.218842 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:06.218785 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47"] Jul 22 12:19:06.219354 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:06.219062 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" containerID="cri-o://05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b" gracePeriod=30 Jul 22 12:19:11.475221 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.475196 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:19:11.577141 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577059 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-socket\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577141 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577109 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-token\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577141 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577133 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-envoy\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577176 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab14aac7-a10d-4167-9c49-7b5b484b7802-istiod-ca-cert\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577199 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-podinfo\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577235 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-certs\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577287 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlx6j\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-kube-api-access-nlx6j\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577305 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-data\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577336 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-credential-socket\") pod \"ab14aac7-a10d-4167-9c49-7b5b484b7802\" (UID: \"ab14aac7-a10d-4167-9c49-7b5b484b7802\") " Jul 22 12:19:11.577529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577359 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:19:11.577929 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577620 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab14aac7-a10d-4167-9c49-7b5b484b7802-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:19:11.577929 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577676 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.577929 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577890 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:19:11.578066 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.577923 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:19:11.578523 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.578501 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:19:11.579616 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.579580 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-kube-api-access-nlx6j" (OuterVolumeSpecName: "kube-api-access-nlx6j") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "kube-api-access-nlx6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:19:11.579616 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.579591 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:19:11.579977 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.579948 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:19:11.580083 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.579954 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab14aac7-a10d-4167-9c49-7b5b484b7802" (UID: "ab14aac7-a10d-4167-9c49-7b5b484b7802"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:19:11.679243 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679212 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679243 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679239 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679254 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlx6j\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-kube-api-access-nlx6j\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679264 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679273 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679281 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679289 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab14aac7-a10d-4167-9c49-7b5b484b7802-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:11.679443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:11.679296 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab14aac7-a10d-4167-9c49-7b5b484b7802-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:19:12.031242 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.031207 2534 generic.go:356] "Generic (PLEG): container finished" podID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerID="05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b" exitCode=0 Jul 22 12:19:12.031416 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.031258 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" Jul 22 12:19:12.031416 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.031254 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" event={"ID":"ab14aac7-a10d-4167-9c49-7b5b484b7802","Type":"ContainerDied","Data":"05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b"} Jul 22 12:19:12.031416 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.031317 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47" event={"ID":"ab14aac7-a10d-4167-9c49-7b5b484b7802","Type":"ContainerDied","Data":"e36a434db410792a8426a86db47a9317ba1d0380ffcf404bac447862f5a31b44"} Jul 22 12:19:12.031416 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.031343 2534 scope.go:117] "RemoveContainer" containerID="05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b" Jul 22 12:19:12.042109 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.042092 2534 scope.go:117] "RemoveContainer" containerID="1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a" Jul 22 12:19:12.050882 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.050866 2534 scope.go:117] "RemoveContainer" containerID="05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b" Jul 22 12:19:12.051140 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:19:12.051120 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b\": container with ID starting with 05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b not found: ID does not exist" containerID="05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b" Jul 22 12:19:12.051245 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.051148 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b"} err="failed to get container status \"05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b\": rpc error: code = NotFound desc = could not find container \"05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b\": container with ID starting with 05ac823bc8a4695329d2bd3f89fb4ad4ba626dac91a90532a5870c422172e56b not found: ID does not exist" Jul 22 12:19:12.051245 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.051166 2534 scope.go:117] "RemoveContainer" containerID="1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a" Jul 22 12:19:12.051428 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:19:12.051408 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a\": container with ID starting with 1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a not found: ID does not exist" containerID="1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a" Jul 22 12:19:12.051508 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.051433 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a"} err="failed to get container status \"1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a\": rpc error: code = NotFound desc = could not find container \"1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a\": container with ID starting with 1ecc19a09c039c3339b92cffec14e543f16f222501cb9bd00dfd2bcd7ebd952a not found: ID does not exist" Jul 22 12:19:12.055492 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.055470 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47"] Jul 22 12:19:12.059329 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.059306 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-94d621ea-openshift-default-7c9ccdf76c-pst47"] Jul 22 12:19:12.102950 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:12.102920 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" path="/var/lib/kubelet/pods/ab14aac7-a10d-4167-9c49-7b5b484b7802/volumes" Jul 22 12:19:32.236408 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:32.236381 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:19:32.239823 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:19:32.239776 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:20:53.640336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640257 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4"] Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640637 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640649 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640674 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640679 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640685 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640691 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640697 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640702 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640759 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640765 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640771 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddee7931-b1d8-485a-afb9-ce22a16ab396" containerName="istio-proxy" Jul 22 12:20:53.640788 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.640779 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c71fa403-94cd-4e90-a2d0-e5371133a0c5" containerName="istio-proxy" Jul 22 12:20:53.644067 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.644048 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.646984 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.646963 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a9b9afab-openshift-default-dockercfg-r4vv7\"" Jul 22 12:20:53.663630 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663598 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istiod-ca-cert\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.663630 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663644 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-data\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.663878 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663696 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-certs\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.663878 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663757 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-token\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.663878 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663825 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-socket\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.664056 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663885 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-podinfo\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.664056 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663935 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-envoy\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.664056 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.663965 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-credential-socket\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.664184 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.664045 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdw4w\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-kube-api-access-vdw4w\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.664986 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.664965 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4"] Jul 22 12:20:53.765288 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765246 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-certs\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765497 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765312 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-token\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765497 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765357 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-socket\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765497 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765374 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-podinfo\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765497 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765402 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-envoy\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765497 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765430 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-credential-socket\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765497 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765463 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdw4w\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-kube-api-access-vdw4w\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765887 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765524 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istiod-ca-cert\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765887 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765558 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-data\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.765887 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765777 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-certs\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.766054 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.765995 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-credential-socket\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.766054 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.766036 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-socket\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.766561 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.766533 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-data\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.766713 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.766558 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istiod-ca-cert\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.768102 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.768073 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-envoy\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.768214 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.768163 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-podinfo\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.777718 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.777693 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-token\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.781016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.780994 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdw4w\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-kube-api-access-vdw4w\") pod \"e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:53.955338 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:53.955260 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:54.120068 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.119756 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4"] Jul 22 12:20:54.121209 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:20:54.121176 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5a5e65_3754_4894_a71b_27f65b14c3e0.slice/crio-e3567f6a782f17144d4b00073141010194a4635bc18b9512f03166d0382eff41 WatchSource:0}: Error finding container e3567f6a782f17144d4b00073141010194a4635bc18b9512f03166d0382eff41: Status 404 returned error can't find the container with id e3567f6a782f17144d4b00073141010194a4635bc18b9512f03166d0382eff41 Jul 22 12:20:54.123965 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.123928 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:20:54.124080 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.124000 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:20:54.124080 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.124028 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:20:54.616713 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.616674 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" event={"ID":"1f5a5e65-3754-4894-a71b-27f65b14c3e0","Type":"ContainerStarted","Data":"ce5d3253b5a68d44e7a612213828de124d0b324c58e4d3a845a3f88cea7f4afc"} Jul 22 12:20:54.616962 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.616743 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" event={"ID":"1f5a5e65-3754-4894-a71b-27f65b14c3e0","Type":"ContainerStarted","Data":"e3567f6a782f17144d4b00073141010194a4635bc18b9512f03166d0382eff41"} Jul 22 12:20:54.640615 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.640549 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podStartSLOduration=1.640534025 podStartE2EDuration="1.640534025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:20:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:20:54.638874857 +0000 UTC m=+1882.964722122" watchObservedRunningTime="2026-07-22 12:20:54.640534025 +0000 UTC m=+1882.966381348" Jul 22 12:20:54.955616 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:54.955588 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:55.961322 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:55.961296 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:56.628561 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:56.628519 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:20:56.629426 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:20:56.629408 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:11.682314 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.681402 2534 patch_prober.go:28] interesting pod/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:36156->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 22 12:21:11.682314 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.681471 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:36156->10.132.0.62:15021: read: connection reset by peer" Jul 22 12:21:11.718156 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.718125 2534 generic.go:356] "Generic (PLEG): container finished" podID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerID="ce5d3253b5a68d44e7a612213828de124d0b324c58e4d3a845a3f88cea7f4afc" exitCode=0 Jul 22 12:21:11.718293 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.718180 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" event={"ID":"1f5a5e65-3754-4894-a71b-27f65b14c3e0","Type":"ContainerDied","Data":"ce5d3253b5a68d44e7a612213828de124d0b324c58e4d3a845a3f88cea7f4afc"} Jul 22 12:21:11.718590 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.718571 2534 scope.go:117] "RemoveContainer" containerID="ce5d3253b5a68d44e7a612213828de124d0b324c58e4d3a845a3f88cea7f4afc" Jul 22 12:21:11.719679 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.719646 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:21:11.719768 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.719717 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:21:11.719768 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:11.719749 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:21:12.729380 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:12.729335 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" event={"ID":"1f5a5e65-3754-4894-a71b-27f65b14c3e0","Type":"ContainerStarted","Data":"e688929969501312b337734dace87631cc7cfd28ef91422851f61490081732e1"} Jul 22 12:21:12.956496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:12.956453 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:12.956747 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:12.956722 2534 patch_prober.go:28] interesting pod/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 12:21:12.956872 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:12.956780 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 12:21:13.956111 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:13.956067 2534 patch_prober.go:28] interesting pod/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 12:21:13.956512 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:13.956133 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 12:21:14.955783 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:14.955744 2534 patch_prober.go:28] interesting pod/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 12:21:14.955966 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:14.955831 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 12:21:15.956193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:15.956157 2534 patch_prober.go:28] interesting pod/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 12:21:15.956594 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:15.956211 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 12:21:16.959699 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:16.959663 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:16.960163 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:16.960043 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:16.960667 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:16.960649 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:30.680787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:30.680754 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4"] Jul 22 12:21:30.681654 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:30.681024 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" containerID="cri-o://e688929969501312b337734dace87631cc7cfd28ef91422851f61490081732e1" gracePeriod=30 Jul 22 12:21:31.962570 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:31.962537 2534 patch_prober.go:28] interesting pod/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 12:21:31.962973 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:31.962591 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 12:21:35.863521 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.863264 2534 generic.go:356] "Generic (PLEG): container finished" podID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerID="e688929969501312b337734dace87631cc7cfd28ef91422851f61490081732e1" exitCode=0 Jul 22 12:21:35.863521 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.863331 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" event={"ID":"1f5a5e65-3754-4894-a71b-27f65b14c3e0","Type":"ContainerDied","Data":"e688929969501312b337734dace87631cc7cfd28ef91422851f61490081732e1"} Jul 22 12:21:35.863521 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.863391 2534 scope.go:117] "RemoveContainer" containerID="ce5d3253b5a68d44e7a612213828de124d0b324c58e4d3a845a3f88cea7f4afc" Jul 22 12:21:35.935671 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.935647 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:35.969651 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969577 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-podinfo\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969651 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969612 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-certs\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969651 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969632 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-data\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969946 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969740 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdw4w\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-kube-api-access-vdw4w\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969946 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969812 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-token\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969946 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969847 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-credential-socket\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969946 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969892 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-envoy\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.969946 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969877 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:21:35.969946 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969931 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istiod-ca-cert\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.970274 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.969983 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-socket\") pod \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\" (UID: \"1f5a5e65-3754-4894-a71b-27f65b14c3e0\") " Jul 22 12:21:35.970339 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.970298 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:35.970499 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.970479 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:21:35.970632 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.970612 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:21:35.970719 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.970653 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:21:35.971120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.971093 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:21:35.972214 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.972190 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-kube-api-access-vdw4w" (OuterVolumeSpecName: "kube-api-access-vdw4w") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "kube-api-access-vdw4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:21:35.972332 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.972222 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:21:35.972555 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.972528 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:21:35.972666 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:35.972553 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f5a5e65-3754-4894-a71b-27f65b14c3e0" (UID: "1f5a5e65-3754-4894-a71b-27f65b14c3e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:21:36.070765 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070734 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.070765 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070763 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.070765 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070772 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.071006 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070782 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.071006 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070814 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdw4w\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-kube-api-access-vdw4w\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.071006 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070823 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.071006 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070831 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.071006 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.070839 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5a5e65-3754-4894-a71b-27f65b14c3e0-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:21:36.869962 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.869933 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" Jul 22 12:21:36.869962 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.869941 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4" event={"ID":"1f5a5e65-3754-4894-a71b-27f65b14c3e0","Type":"ContainerDied","Data":"e3567f6a782f17144d4b00073141010194a4635bc18b9512f03166d0382eff41"} Jul 22 12:21:36.870456 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.870013 2534 scope.go:117] "RemoveContainer" containerID="e688929969501312b337734dace87631cc7cfd28ef91422851f61490081732e1" Jul 22 12:21:36.892195 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.892166 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4"] Jul 22 12:21:36.903067 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:36.903044 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a9b9afab-openshift-default-5d9b67c555-lhnv4"] Jul 22 12:21:38.101649 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:38.101614 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" path="/var/lib/kubelet/pods/1f5a5e65-3754-4894-a71b-27f65b14c3e0/volumes" Jul 22 12:21:50.734435 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.734403 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh"] Jul 22 12:21:50.734954 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.734937 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" Jul 22 12:21:50.735003 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.734956 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" Jul 22 12:21:50.735003 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.734972 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" Jul 22 12:21:50.735003 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.734977 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" Jul 22 12:21:50.735105 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.735049 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" Jul 22 12:21:50.735105 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.735062 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab14aac7-a10d-4167-9c49-7b5b484b7802" containerName="istio-proxy" Jul 22 12:21:50.739589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.739573 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.742017 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.741994 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8cd00702-openshift-default-dockercfg-thd9t\"" Jul 22 12:21:50.751006 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.750955 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh"] Jul 22 12:21:50.805660 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805634 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnw24\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-kube-api-access-lnw24\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.805825 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805669 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-token\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.805825 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805694 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-credential-socket\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.805825 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805782 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-podinfo\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.805964 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805845 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-envoy\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.805964 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805870 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istiod-ca-cert\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.805964 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805893 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-data\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.806091 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.805977 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-certs\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.806091 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.806018 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-socket\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.906980 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnw24\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-kube-api-access-lnw24\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907022 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-token\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907049 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-credential-socket\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907091 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-podinfo\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907117 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-envoy\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907143 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istiod-ca-cert\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907171 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-data\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907228 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-certs\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907271 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-socket\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907727 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907473 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-credential-socket\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907727 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907594 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-data\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907727 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907623 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-certs\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907727 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907686 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-socket\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.907933 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.907853 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istiod-ca-cert\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.909527 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.909508 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-envoy\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.909842 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.909824 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-podinfo\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.915915 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.915892 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-token\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:50.916217 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:50.916198 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnw24\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-kube-api-access-lnw24\") pod \"e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:51.051070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.050989 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:51.196103 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.196070 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh"] Jul 22 12:21:51.197658 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:21:51.197624 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1f4e53_8ceb_4971_8630_e15aaa654ea5.slice/crio-2270d915cea51758edb8154f316b11836bd6d4557453f3697885ef459881b935 WatchSource:0}: Error finding container 2270d915cea51758edb8154f316b11836bd6d4557453f3697885ef459881b935: Status 404 returned error can't find the container with id 2270d915cea51758edb8154f316b11836bd6d4557453f3697885ef459881b935 Jul 22 12:21:51.199928 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.199888 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:21:51.199997 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.199978 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:21:51.200039 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.200028 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:21:51.957918 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.957879 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" event={"ID":"fe1f4e53-8ceb-4971-8630-e15aaa654ea5","Type":"ContainerStarted","Data":"01b516958236ffb8edf35cad6740bb7586bc29110dd8ab9613aa457dfae74cea"} Jul 22 12:21:51.958316 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.957977 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" event={"ID":"fe1f4e53-8ceb-4971-8630-e15aaa654ea5","Type":"ContainerStarted","Data":"2270d915cea51758edb8154f316b11836bd6d4557453f3697885ef459881b935"} Jul 22 12:21:51.980029 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:51.979979 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" podStartSLOduration=1.979962071 podStartE2EDuration="1.979962071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:21:51.978321651 +0000 UTC m=+1940.304168914" watchObservedRunningTime="2026-07-22 12:21:51.979962071 +0000 UTC m=+1940.305809407" Jul 22 12:21:52.051976 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:52.051939 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:52.057441 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:52.057411 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:52.964065 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:52.964029 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:21:52.964927 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:21:52.964900 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:22:07.966056 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:07.966016 2534 patch_prober.go:28] interesting pod/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 12:22:07.966471 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:07.966072 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 12:22:08.060689 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:08.060653 2534 generic.go:356] "Generic (PLEG): container finished" podID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerID="01b516958236ffb8edf35cad6740bb7586bc29110dd8ab9613aa457dfae74cea" exitCode=0 Jul 22 12:22:08.060879 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:08.060726 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" event={"ID":"fe1f4e53-8ceb-4971-8630-e15aaa654ea5","Type":"ContainerDied","Data":"01b516958236ffb8edf35cad6740bb7586bc29110dd8ab9613aa457dfae74cea"} Jul 22 12:22:08.061192 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:08.061177 2534 scope.go:117] "RemoveContainer" containerID="01b516958236ffb8edf35cad6740bb7586bc29110dd8ab9613aa457dfae74cea" Jul 22 12:22:08.062173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:08.062146 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:22:08.062281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:08.062209 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:22:08.062281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:08.062245 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:22:09.067899 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:09.067830 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" event={"ID":"fe1f4e53-8ceb-4971-8630-e15aaa654ea5","Type":"ContainerStarted","Data":"14885fb340b600b05a1a465130798712bf16825ffc54644eb9b99b5d72779826"} Jul 22 12:22:10.051757 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:10.051727 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:22:10.051991 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:10.051974 2534 patch_prober.go:28] interesting pod/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 12:22:10.052043 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:10.052015 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 12:22:11.052307 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:11.052273 2534 patch_prober.go:28] interesting pod/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 12:22:11.052715 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:11.052326 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 12:22:12.051742 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:12.051709 2534 patch_prober.go:28] interesting pod/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 12:22:12.051962 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:12.051760 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 12:22:13.056172 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:13.056142 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:22:13.056529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:13.056394 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:22:13.057141 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:22:13.057121 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:23:48.374982 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:48.374946 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh"] Jul 22 12:23:48.375496 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:48.375182 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" containerID="cri-o://14885fb340b600b05a1a465130798712bf16825ffc54644eb9b99b5d72779826" gracePeriod=30 Jul 22 12:23:53.634341 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.634271 2534 generic.go:356] "Generic (PLEG): container finished" podID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerID="14885fb340b600b05a1a465130798712bf16825ffc54644eb9b99b5d72779826" exitCode=0 Jul 22 12:23:53.634956 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.634334 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" event={"ID":"fe1f4e53-8ceb-4971-8630-e15aaa654ea5","Type":"ContainerDied","Data":"14885fb340b600b05a1a465130798712bf16825ffc54644eb9b99b5d72779826"} Jul 22 12:23:53.634956 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.634392 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" event={"ID":"fe1f4e53-8ceb-4971-8630-e15aaa654ea5","Type":"ContainerDied","Data":"2270d915cea51758edb8154f316b11836bd6d4557453f3697885ef459881b935"} Jul 22 12:23:53.634956 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.634409 2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2270d915cea51758edb8154f316b11836bd6d4557453f3697885ef459881b935" Jul 22 12:23:53.634956 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.634436 2534 scope.go:117] "RemoveContainer" containerID="01b516958236ffb8edf35cad6740bb7586bc29110dd8ab9613aa457dfae74cea" Jul 22 12:23:53.640682 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.640648 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:23:53.768112 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768078 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-podinfo\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768112 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768119 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-socket\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768473 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768143 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-data\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768473 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768329 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-credential-socket\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768473 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768352 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:23:53.768473 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768406 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-envoy\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768473 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768451 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-certs\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768745 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768490 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-token\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768745 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768521 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnw24\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-kube-api-access-lnw24\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768745 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768541 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:23:53.768745 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768572 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istiod-ca-cert\") pod \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\" (UID: \"fe1f4e53-8ceb-4971-8630-e15aaa654ea5\") " Jul 22 12:23:53.768745 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768723 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:23:53.769052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768866 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:23:53.769052 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.768928 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:23:53.769173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.769069 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.769173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.769086 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.769173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.769100 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.769173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.769114 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.769173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.769126 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.770705 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.770668 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:23:53.771213 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.771181 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:23:53.771640 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.771610 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:23:53.771640 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.771626 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-kube-api-access-lnw24" (OuterVolumeSpecName: "kube-api-access-lnw24") pod "fe1f4e53-8ceb-4971-8630-e15aaa654ea5" (UID: "fe1f4e53-8ceb-4971-8630-e15aaa654ea5"). InnerVolumeSpecName "kube-api-access-lnw24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:23:53.870506 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.870470 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.870506 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.870500 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.870506 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.870509 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnw24\" (UniqueName: \"kubernetes.io/projected/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-kube-api-access-lnw24\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:53.870736 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:53.870520 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe1f4e53-8ceb-4971-8630-e15aaa654ea5-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:23:54.642237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:54.642209 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh" Jul 22 12:23:54.708398 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:54.708365 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh"] Jul 22 12:23:54.712067 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:54.712036 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8cd00702-openshift-default-5c8bdb9787-brszh"] Jul 22 12:23:56.106480 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:23:56.106445 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" path="/var/lib/kubelet/pods/fe1f4e53-8ceb-4971-8630-e15aaa654ea5/volumes" Jul 22 12:24:32.273119 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:32.273094 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:24:32.277345 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:32.277324 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:24:46.284315 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284285 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84"] Jul 22 12:24:46.284681 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284651 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" Jul 22 12:24:46.284681 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284661 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" Jul 22 12:24:46.284681 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284679 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" Jul 22 12:24:46.284787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284684 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" Jul 22 12:24:46.284787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284741 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" Jul 22 12:24:46.284787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284751 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5a5e65-3754-4894-a71b-27f65b14c3e0" containerName="istio-proxy" Jul 22 12:24:46.284787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.284759 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe1f4e53-8ceb-4971-8630-e15aaa654ea5" containerName="istio-proxy" Jul 22 12:24:46.287725 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.287709 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.290188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.290168 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-53f570-openshift-default-dockercfg-vs2m7\"" Jul 22 12:24:46.298577 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.298554 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84"] Jul 22 12:24:46.349390 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349360 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istiod-ca-cert\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349394 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-token\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349425 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-data\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349485 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-socket\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349529 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349516 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-credential-socket\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349668 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349544 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96dn4\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-kube-api-access-96dn4\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349668 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349608 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-envoy\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349668 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349625 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-podinfo\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.349760 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.349672 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-certs\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450274 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450238 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-data\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450274 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450279 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-socket\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450300 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-credential-socket\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450327 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96dn4\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-kube-api-access-96dn4\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450371 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-envoy\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450390 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-podinfo\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450431 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-certs\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450468 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istiod-ca-cert\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450538 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450497 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-token\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450949 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450723 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-socket\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450949 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450903 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-credential-socket\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.450949 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.450936 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-data\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.451095 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.451013 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-certs\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.451310 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.451274 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istiod-ca-cert\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.452929 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.452903 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-envoy\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.453049 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.453025 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-podinfo\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.459653 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.459630 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-token\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.459895 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.459877 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96dn4\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-kube-api-access-96dn4\") pod \"e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.599565 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.599487 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:46.742560 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.742530 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84"] Jul 22 12:24:46.744225 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:24:46.744194 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode427373f_81e5_4a9c_a2ae_fc84be5fdd98.slice/crio-52c558f3953cc9bfd30856be26ac1e2730316bfc4339af736af51e5af45700d1 WatchSource:0}: Error finding container 52c558f3953cc9bfd30856be26ac1e2730316bfc4339af736af51e5af45700d1: Status 404 returned error can't find the container with id 52c558f3953cc9bfd30856be26ac1e2730316bfc4339af736af51e5af45700d1 Jul 22 12:24:46.746327 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.746296 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:24:46.746421 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.746362 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:24:46.746421 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.746391 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:24:46.955581 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.955540 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" event={"ID":"e427373f-81e5-4a9c-a2ae-fc84be5fdd98","Type":"ContainerStarted","Data":"532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c"} Jul 22 12:24:46.955758 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.955637 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" event={"ID":"e427373f-81e5-4a9c-a2ae-fc84be5fdd98","Type":"ContainerStarted","Data":"52c558f3953cc9bfd30856be26ac1e2730316bfc4339af736af51e5af45700d1"} Jul 22 12:24:46.976844 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:46.976763 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podStartSLOduration=0.976742193 podStartE2EDuration="976.742193ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:24:46.975272428 +0000 UTC m=+2115.301119748" watchObservedRunningTime="2026-07-22 12:24:46.976742193 +0000 UTC m=+2115.302589458" Jul 22 12:24:47.600654 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:47.600615 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:47.605943 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:47.605916 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:47.960570 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:47.960542 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:47.961635 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:47.961616 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:24:54.079568 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.079511 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km"] Jul 22 12:24:54.084839 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.084814 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.087071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.087052 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ffe082-openshift-default-dockercfg-fxbb2\"" Jul 22 12:24:54.094808 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.094769 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km"] Jul 22 12:24:54.227922 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.227890 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874c1352-0514-4186-817b-5ae42094b11f-istio-podinfo\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.227922 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.227932 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-data\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.227959 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wstfm\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-kube-api-access-wstfm\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.228029 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-istio-token\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.228063 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-certs\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.228083 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874c1352-0514-4186-817b-5ae42094b11f-istiod-ca-cert\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.228103 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-socket\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.228161 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-envoy\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.228412 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.228259 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-credential-socket\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329190 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329159 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874c1352-0514-4186-817b-5ae42094b11f-istio-podinfo\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329204 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-data\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329231 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wstfm\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-kube-api-access-wstfm\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329258 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-istio-token\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329293 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-certs\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329309 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874c1352-0514-4186-817b-5ae42094b11f-istiod-ca-cert\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329657 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329399 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-socket\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329657 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329518 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-envoy\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329657 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329612 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-credential-socket\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329657 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329632 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-data\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.329924 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329776 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-socket\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.330012 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.329990 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-certs\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.330080 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.330005 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-credential-socket\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.330249 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.330227 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874c1352-0514-4186-817b-5ae42094b11f-istiod-ca-cert\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.331828 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.331782 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-envoy\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.331939 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.331837 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874c1352-0514-4186-817b-5ae42094b11f-istio-podinfo\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.338221 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.338194 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wstfm\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-kube-api-access-wstfm\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.338310 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.338205 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-istio-token\") pod \"e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.396945 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.396913 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:54.548762 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.548734 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km"] Jul 22 12:24:54.549391 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:24:54.549368 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod874c1352_0514_4186_817b_5ae42094b11f.slice/crio-bee1f06215be4681238ee44fd6f2dbb3d02b425bb8118e55dcdfe1dc7ee4e733 WatchSource:0}: Error finding container bee1f06215be4681238ee44fd6f2dbb3d02b425bb8118e55dcdfe1dc7ee4e733: Status 404 returned error can't find the container with id bee1f06215be4681238ee44fd6f2dbb3d02b425bb8118e55dcdfe1dc7ee4e733 Jul 22 12:24:54.551607 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.551574 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:24:54.551685 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.551638 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:24:54.551685 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:54.551669 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:24:55.001040 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:55.001005 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" event={"ID":"874c1352-0514-4186-817b-5ae42094b11f","Type":"ContainerStarted","Data":"f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e"} Jul 22 12:24:55.001197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:55.001063 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" event={"ID":"874c1352-0514-4186-817b-5ae42094b11f","Type":"ContainerStarted","Data":"bee1f06215be4681238ee44fd6f2dbb3d02b425bb8118e55dcdfe1dc7ee4e733"} Jul 22 12:24:55.022926 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:55.022871 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" podStartSLOduration=1.02285564 podStartE2EDuration="1.02285564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:24:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:24:55.02066059 +0000 UTC m=+2123.346507853" watchObservedRunningTime="2026-07-22 12:24:55.02285564 +0000 UTC m=+2123.348702905" Jul 22 12:24:55.398118 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:55.398083 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:55.402461 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:55.402438 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:56.006857 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:56.006828 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:24:56.008049 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:24:56.008029 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:25:02.962162 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:02.962074 2534 patch_prober.go:28] interesting pod/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 12:25:02.962162 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:02.962141 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 12:25:03.058685 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:03.058652 2534 generic.go:356] "Generic (PLEG): container finished" podID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerID="532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c" exitCode=0 Jul 22 12:25:03.058867 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:03.058703 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" event={"ID":"e427373f-81e5-4a9c-a2ae-fc84be5fdd98","Type":"ContainerDied","Data":"532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c"} Jul 22 12:25:03.059109 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:03.059096 2534 scope.go:117] "RemoveContainer" containerID="532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c" Jul 22 12:25:03.060120 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:03.060090 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:25:03.060244 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:03.060160 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:25:03.060244 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:03.060193 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:25:04.071274 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:04.071235 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" event={"ID":"e427373f-81e5-4a9c-a2ae-fc84be5fdd98","Type":"ContainerStarted","Data":"ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4"} Jul 22 12:25:04.600615 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:04.600577 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:25:04.600832 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:04.600810 2534 patch_prober.go:28] interesting pod/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 12:25:04.600886 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:04.600857 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 12:25:05.600867 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:05.600831 2534 patch_prober.go:28] interesting pod/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 12:25:05.601280 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:05.600901 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 12:25:06.600246 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:06.600213 2534 patch_prober.go:28] interesting pod/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 12:25:06.600421 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:06.600260 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 12:25:07.600874 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:07.600830 2534 patch_prober.go:28] interesting pod/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 12:25:07.601408 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:07.600900 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 12:25:08.604470 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:08.604438 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:25:08.605004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:08.604733 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:25:08.605397 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:08.605379 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:25:10.109762 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:10.109674 2534 generic.go:356] "Generic (PLEG): container finished" podID="874c1352-0514-4186-817b-5ae42094b11f" containerID="f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e" exitCode=0 Jul 22 12:25:10.109762 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:10.109742 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" event={"ID":"874c1352-0514-4186-817b-5ae42094b11f","Type":"ContainerDied","Data":"f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e"} Jul 22 12:25:10.110275 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:10.110259 2534 scope.go:117] "RemoveContainer" containerID="f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e" Jul 22 12:25:10.111290 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:10.111259 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:25:10.111376 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:10.111329 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:25:10.111376 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:10.111360 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:25:11.006909 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:11.006876 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:25:11.121904 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:11.121868 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" event={"ID":"874c1352-0514-4186-817b-5ae42094b11f","Type":"ContainerStarted","Data":"ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca"} Jul 22 12:25:11.397275 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:11.397241 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:25:11.398577 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:11.398550 2534 patch_prober.go:28] interesting pod/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 12:25:11.398701 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:11.398600 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 12:25:12.397631 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:12.397595 2534 patch_prober.go:28] interesting pod/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 12:25:12.398077 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:12.397664 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 12:25:13.398300 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:13.398261 2534 patch_prober.go:28] interesting pod/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 12:25:13.398693 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:13.398333 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 12:25:14.397818 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:14.397754 2534 patch_prober.go:28] interesting pod/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 12:25:14.398000 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:14.397850 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 12:25:15.400965 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:15.400930 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:25:15.401353 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:15.401294 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:25:15.401919 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:25:15.401900 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:26:39.710551 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:39.710515 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84"] Jul 22 12:26:39.711167 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:39.710717 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" containerID="cri-o://ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4" gracePeriod=30 Jul 22 12:26:44.962815 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:44.962765 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:26:45.110939 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.110850 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-token\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.110939 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.110892 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-envoy\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.110939 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.110914 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-credential-socket\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.110939 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.110931 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istiod-ca-cert\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.111281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.110962 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-socket\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.111281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.110990 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-data\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.111281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111058 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-podinfo\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.111281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111111 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96dn4\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-kube-api-access-96dn4\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.111281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111159 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-certs\") pod \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\" (UID: \"e427373f-81e5-4a9c-a2ae-fc84be5fdd98\") " Jul 22 12:26:45.111281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111223 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:26:45.111597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111285 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:26:45.111597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111321 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:26:45.111597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111483 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.111597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111490 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:26:45.111597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111501 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.111597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.111514 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.112319 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.112287 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-data" (OuterVolumeSpecName: "istio-data") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:26:45.113325 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.113295 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:26:45.113435 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.113373 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-kube-api-access-96dn4" (OuterVolumeSpecName: "kube-api-access-96dn4") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "kube-api-access-96dn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:26:45.113485 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.113436 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-token" (OuterVolumeSpecName: "istio-token") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:26:45.113525 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.113480 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e427373f-81e5-4a9c-a2ae-fc84be5fdd98" (UID: "e427373f-81e5-4a9c-a2ae-fc84be5fdd98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:26:45.212227 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.212187 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.212227 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.212221 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.212227 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.212233 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.212458 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.212244 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.212458 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.212256 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96dn4\" (UniqueName: \"kubernetes.io/projected/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-kube-api-access-96dn4\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.212458 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.212269 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e427373f-81e5-4a9c-a2ae-fc84be5fdd98-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:26:45.672068 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.672036 2534 generic.go:356] "Generic (PLEG): container finished" podID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerID="ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4" exitCode=0 Jul 22 12:26:45.672285 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.672087 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" Jul 22 12:26:45.672285 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.672098 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" event={"ID":"e427373f-81e5-4a9c-a2ae-fc84be5fdd98","Type":"ContainerDied","Data":"ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4"} Jul 22 12:26:45.672285 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.672164 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84" event={"ID":"e427373f-81e5-4a9c-a2ae-fc84be5fdd98","Type":"ContainerDied","Data":"52c558f3953cc9bfd30856be26ac1e2730316bfc4339af736af51e5af45700d1"} Jul 22 12:26:45.672285 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.672193 2534 scope.go:117] "RemoveContainer" containerID="ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4" Jul 22 12:26:45.685574 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.683786 2534 scope.go:117] "RemoveContainer" containerID="532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c" Jul 22 12:26:45.694467 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.694449 2534 scope.go:117] "RemoveContainer" containerID="ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4" Jul 22 12:26:45.694738 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:26:45.694719 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4\": container with ID starting with ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4 not found: ID does not exist" containerID="ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4" Jul 22 12:26:45.694827 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.694747 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4"} err="failed to get container status \"ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4\": rpc error: code = NotFound desc = could not find container \"ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4\": container with ID starting with ab95698d4b6bd8dd0e5b104c7b9d247b521dd031cbf477fc764dc212ac16a6c4 not found: ID does not exist" Jul 22 12:26:45.694827 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.694765 2534 scope.go:117] "RemoveContainer" containerID="532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c" Jul 22 12:26:45.695033 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:26:45.695017 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c\": container with ID starting with 532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c not found: ID does not exist" containerID="532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c" Jul 22 12:26:45.695085 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.695038 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c"} err="failed to get container status \"532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c\": rpc error: code = NotFound desc = could not find container \"532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c\": container with ID starting with 532c48680c4522e1a4dd3b0a4951758b5cfd46880b3373d6931d4cf5d3d5cd0c not found: ID does not exist" Jul 22 12:26:45.697168 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.697149 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84"] Jul 22 12:26:45.701455 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:45.701432 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-53f570-openshift-default-7d7c56c65-f6x84"] Jul 22 12:26:46.101367 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:26:46.101286 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" path="/var/lib/kubelet/pods/e427373f-81e5-4a9c-a2ae-fc84be5fdd98/volumes" Jul 22 12:27:02.353523 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:02.353447 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km"] Jul 22 12:27:02.353936 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:02.353685 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" containerID="cri-o://ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca" gracePeriod=30 Jul 22 12:27:03.247641 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.247604 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7"] Jul 22 12:27:03.248222 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.248199 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" Jul 22 12:27:03.248284 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.248226 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" Jul 22 12:27:03.248361 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.248350 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" Jul 22 12:27:03.248401 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.248366 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" Jul 22 12:27:03.253211 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.253193 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.256060 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.256037 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e96939-openshift-default-dockercfg-fc49z\"" Jul 22 12:27:03.263035 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.263008 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7"] Jul 22 12:27:03.381686 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381651 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-certs\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381711 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-socket\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381825 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-token\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381869 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-envoy\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381918 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-data\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381940 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-podinfo\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.381983 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b80be94-ac35-4d65-92a9-d836d01f58ed-istiod-ca-cert\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.382005 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4v6h\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-kube-api-access-m4v6h\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.382173 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.382065 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-credential-socket\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483484 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483449 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-socket\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483654 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483496 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-token\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483654 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483610 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-envoy\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483654 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483646 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-data\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483786 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483663 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-podinfo\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483786 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483685 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b80be94-ac35-4d65-92a9-d836d01f58ed-istiod-ca-cert\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483786 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483715 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4v6h\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-kube-api-access-m4v6h\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483974 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483845 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-credential-socket\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483974 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483878 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-certs\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.483974 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.483928 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-socket\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.484137 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.484040 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-data\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.484219 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.484201 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-certs\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.484277 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.484241 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-credential-socket\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.484503 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.484475 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b80be94-ac35-4d65-92a9-d836d01f58ed-istiod-ca-cert\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.486087 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.486066 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-envoy\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.486396 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.486372 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-podinfo\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.493425 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.493402 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-token\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.494092 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.494072 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4v6h\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-kube-api-access-m4v6h\") pod \"e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.565108 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.565021 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:03.708059 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.708028 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7"] Jul 22 12:27:03.709599 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:27:03.709567 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b80be94_ac35_4d65_92a9_d836d01f58ed.slice/crio-884bba4b0733245728e07b480a8dc3728eeb6ef5ef22e6f1dc69f614957ac890 WatchSource:0}: Error finding container 884bba4b0733245728e07b480a8dc3728eeb6ef5ef22e6f1dc69f614957ac890: Status 404 returned error can't find the container with id 884bba4b0733245728e07b480a8dc3728eeb6ef5ef22e6f1dc69f614957ac890 Jul 22 12:27:03.711896 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.711861 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:03.711990 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.711926 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:03.711990 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.711955 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:03.775774 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:03.775750 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" event={"ID":"0b80be94-ac35-4d65-92a9-d836d01f58ed","Type":"ContainerStarted","Data":"884bba4b0733245728e07b480a8dc3728eeb6ef5ef22e6f1dc69f614957ac890"} Jul 22 12:27:04.782778 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:04.782738 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" event={"ID":"0b80be94-ac35-4d65-92a9-d836d01f58ed","Type":"ContainerStarted","Data":"69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8"} Jul 22 12:27:04.807582 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:04.807534 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podStartSLOduration=1.8075145030000002 podStartE2EDuration="1.807514503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:27:04.804728482 +0000 UTC m=+2253.130575746" watchObservedRunningTime="2026-07-22 12:27:04.807514503 +0000 UTC m=+2253.133361766" Jul 22 12:27:05.565297 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:05.565262 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:05.570483 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:05.570460 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:05.788736 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:05.788710 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:05.789787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:05.789771 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:07.610812 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.610771 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:27:07.619914 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.619857 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-socket\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.619914 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.619899 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874c1352-0514-4186-817b-5ae42094b11f-istiod-ca-cert\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.619953 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874c1352-0514-4186-817b-5ae42094b11f-istio-podinfo\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.619973 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-istio-token\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620005 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-certs\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620032 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-credential-socket\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620680 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620075 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-data\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620680 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620600 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wstfm\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-kube-api-access-wstfm\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620680 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620658 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-envoy\") pod \"874c1352-0514-4186-817b-5ae42094b11f\" (UID: \"874c1352-0514-4186-817b-5ae42094b11f\") " Jul 22 12:27:07.620926 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620724 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:07.620996 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620741 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/874c1352-0514-4186-817b-5ae42094b11f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:27:07.621395 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.621237 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-data" (OuterVolumeSpecName: "istio-data") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:07.621395 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.620139 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:07.621566 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.621475 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:07.623726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.623676 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/874c1352-0514-4186-817b-5ae42094b11f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:27:07.623726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.623684 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-istio-token" (OuterVolumeSpecName: "istio-token") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:27:07.623933 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.623754 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-kube-api-access-wstfm" (OuterVolumeSpecName: "kube-api-access-wstfm") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "kube-api-access-wstfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:27:07.629707 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.629679 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.629934 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.629752 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/874c1352-0514-4186-817b-5ae42094b11f-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.629934 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.629920 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.630051 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.629939 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.630134 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.630110 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.631735 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.631670 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "874c1352-0514-4186-817b-5ae42094b11f" (UID: "874c1352-0514-4186-817b-5ae42094b11f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:07.731210 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.731174 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wstfm\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-kube-api-access-wstfm\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.731210 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.731206 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/874c1352-0514-4186-817b-5ae42094b11f-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.731401 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.731220 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/874c1352-0514-4186-817b-5ae42094b11f-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.731401 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.731231 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/874c1352-0514-4186-817b-5ae42094b11f-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:07.802227 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.802188 2534 generic.go:356] "Generic (PLEG): container finished" podID="874c1352-0514-4186-817b-5ae42094b11f" containerID="ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca" exitCode=0 Jul 22 12:27:07.802404 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.802258 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" Jul 22 12:27:07.802404 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.802280 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" event={"ID":"874c1352-0514-4186-817b-5ae42094b11f","Type":"ContainerDied","Data":"ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca"} Jul 22 12:27:07.802404 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.802357 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km" event={"ID":"874c1352-0514-4186-817b-5ae42094b11f","Type":"ContainerDied","Data":"bee1f06215be4681238ee44fd6f2dbb3d02b425bb8118e55dcdfe1dc7ee4e733"} Jul 22 12:27:07.802404 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.802382 2534 scope.go:117] "RemoveContainer" containerID="ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca" Jul 22 12:27:07.812650 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.812634 2534 scope.go:117] "RemoveContainer" containerID="f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e" Jul 22 12:27:07.822377 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.822361 2534 scope.go:117] "RemoveContainer" containerID="ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca" Jul 22 12:27:07.822640 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:27:07.822622 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca\": container with ID starting with ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca not found: ID does not exist" containerID="ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca" Jul 22 12:27:07.822690 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.822649 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca"} err="failed to get container status \"ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca\": rpc error: code = NotFound desc = could not find container \"ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca\": container with ID starting with ddc5b307e417470013b30741f6818acbc788a95bb4f1e071706a1945c7b60fca not found: ID does not exist" Jul 22 12:27:07.822690 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.822667 2534 scope.go:117] "RemoveContainer" containerID="f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e" Jul 22 12:27:07.822921 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:27:07.822903 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e\": container with ID starting with f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e not found: ID does not exist" containerID="f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e" Jul 22 12:27:07.822967 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.822927 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e"} err="failed to get container status \"f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e\": rpc error: code = NotFound desc = could not find container \"f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e\": container with ID starting with f93213eb4880718e105a355c0880fb0d3133a1249c577eff5efebadc04ea9c7e not found: ID does not exist" Jul 22 12:27:07.829898 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.829876 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km"] Jul 22 12:27:07.837085 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:07.837065 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ffe082-openshift-default-5d6d5cb545-dm5km"] Jul 22 12:27:08.101115 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:08.101084 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="874c1352-0514-4186-817b-5ae42094b11f" path="/var/lib/kubelet/pods/874c1352-0514-4186-817b-5ae42094b11f/volumes" Jul 22 12:27:11.968121 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968087 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv"] Jul 22 12:27:11.968589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968475 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" Jul 22 12:27:11.968589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968486 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427373f-81e5-4a9c-a2ae-fc84be5fdd98" containerName="istio-proxy" Jul 22 12:27:11.968589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968494 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" Jul 22 12:27:11.968589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968500 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" Jul 22 12:27:11.968589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968523 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" Jul 22 12:27:11.968589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968532 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" Jul 22 12:27:11.968778 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968605 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" Jul 22 12:27:11.968778 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.968614 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="874c1352-0514-4186-817b-5ae42094b11f" containerName="istio-proxy" Jul 22 12:27:11.972516 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.972499 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:11.975156 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.975133 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9edc58-openshift-default-dockercfg-dw82d\"" Jul 22 12:27:11.984640 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:11.984618 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv"] Jul 22 12:27:12.067670 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.067634 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-istio-token\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068005 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.067958 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4460350e-e753-418f-8164-2d7c46f67703-istio-podinfo\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068151 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068050 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-data\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068151 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068123 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4460350e-e753-418f-8164-2d7c46f67703-istiod-ca-cert\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068263 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068148 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-credential-socket\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068263 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068184 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-socket\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068263 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068216 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-envoy\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068263 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068250 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prbkc\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-kube-api-access-prbkc\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.068464 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.068279 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-certs\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.169557 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169522 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4460350e-e753-418f-8164-2d7c46f67703-istiod-ca-cert\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.169557 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169558 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-credential-socket\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.169872 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169595 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-socket\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.169872 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169616 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-envoy\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.169872 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169635 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prbkc\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-kube-api-access-prbkc\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170042 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169867 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-certs\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170042 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.169945 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-istio-token\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170042 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170001 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4460350e-e753-418f-8164-2d7c46f67703-istio-podinfo\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170203 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170033 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-credential-socket\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170203 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170061 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-data\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170203 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170063 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-socket\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170600 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170199 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-certs\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170780 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170602 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-data\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.170917 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.170776 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4460350e-e753-418f-8164-2d7c46f67703-istiod-ca-cert\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.177824 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.174245 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-envoy\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.177824 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.174818 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4460350e-e753-418f-8164-2d7c46f67703-istio-podinfo\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.179966 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.179945 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prbkc\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-kube-api-access-prbkc\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.180248 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.180223 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-istio-token\") pod \"e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.283890 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.283785 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:12.430376 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.430347 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv"] Jul 22 12:27:12.432324 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:27:12.432293 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4460350e_e753_418f_8164_2d7c46f67703.slice/crio-64dff689ce94c225d6904ea2a8f6279968b63ed17d3ec2c97f23e69b1a36678e WatchSource:0}: Error finding container 64dff689ce94c225d6904ea2a8f6279968b63ed17d3ec2c97f23e69b1a36678e: Status 404 returned error can't find the container with id 64dff689ce94c225d6904ea2a8f6279968b63ed17d3ec2c97f23e69b1a36678e Jul 22 12:27:12.434378 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.434343 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:12.434466 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.434413 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:12.434466 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.434440 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:12.837776 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.837741 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" event={"ID":"4460350e-e753-418f-8164-2d7c46f67703","Type":"ContainerStarted","Data":"6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a"} Jul 22 12:27:12.837996 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.837817 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" event={"ID":"4460350e-e753-418f-8164-2d7c46f67703","Type":"ContainerStarted","Data":"64dff689ce94c225d6904ea2a8f6279968b63ed17d3ec2c97f23e69b1a36678e"} Jul 22 12:27:12.860728 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:12.860676 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podStartSLOduration=1.860660342 podStartE2EDuration="1.860660342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:27:12.858131035 +0000 UTC m=+2261.183978336" watchObservedRunningTime="2026-07-22 12:27:12.860660342 +0000 UTC m=+2261.186507605" Jul 22 12:27:13.284868 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:13.284830 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:13.289485 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:13.289461 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:13.844096 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:13.844061 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:13.845126 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:13.845108 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:20.906669 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:20.905564 2534 patch_prober.go:28] interesting pod/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": read tcp 10.132.0.2:40158->10.132.0.66:15021: read: connection reset by peer" start-of-body= Jul 22 12:27:20.906669 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:20.905645 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": read tcp 10.132.0.2:40158->10.132.0.66:15021: read: connection reset by peer" Jul 22 12:27:21.896659 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:21.896627 2534 generic.go:356] "Generic (PLEG): container finished" podID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerID="69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8" exitCode=0 Jul 22 12:27:21.896881 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:21.896703 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" event={"ID":"0b80be94-ac35-4d65-92a9-d836d01f58ed","Type":"ContainerDied","Data":"69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8"} Jul 22 12:27:21.897165 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:21.897152 2534 scope.go:117] "RemoveContainer" containerID="69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8" Jul 22 12:27:21.898237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:21.898205 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:21.898302 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:21.898286 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:21.898354 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:21.898324 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:22.904589 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:22.904552 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" event={"ID":"0b80be94-ac35-4d65-92a9-d836d01f58ed","Type":"ContainerStarted","Data":"b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962"} Jul 22 12:27:23.566406 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:23.566365 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:23.566597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:23.566485 2534 patch_prober.go:28] interesting pod/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 12:27:23.566597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:23.566534 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 12:27:24.566262 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:24.566217 2534 patch_prober.go:28] interesting pod/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 12:27:24.566777 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:24.566285 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 12:27:25.566014 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:25.565977 2534 patch_prober.go:28] interesting pod/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 12:27:25.566195 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:25.566034 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 12:27:26.565729 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:26.565687 2534 patch_prober.go:28] interesting pod/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 12:27:26.566161 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:26.565760 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 12:27:27.569218 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:27.569180 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:27.569706 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:27.569463 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:27.570281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:27.570263 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:28.845438 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.845401 2534 patch_prober.go:28] interesting pod/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 12:27:28.845907 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.845455 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 12:27:28.952743 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.952708 2534 generic.go:356] "Generic (PLEG): container finished" podID="4460350e-e753-418f-8164-2d7c46f67703" containerID="6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a" exitCode=0 Jul 22 12:27:28.952948 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.952782 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" event={"ID":"4460350e-e753-418f-8164-2d7c46f67703","Type":"ContainerDied","Data":"6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a"} Jul 22 12:27:28.953336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.953316 2534 scope.go:117] "RemoveContainer" containerID="6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a" Jul 22 12:27:28.954323 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.954297 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:28.954381 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.954366 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:28.954423 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:28.954408 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:27:29.966082 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:29.966039 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" event={"ID":"4460350e-e753-418f-8164-2d7c46f67703","Type":"ContainerStarted","Data":"91d4c66b6f8d02184e234b2239f3dc5be21f712496372d5403fa6ef0965416f4"} Jul 22 12:27:30.284868 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:30.284745 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:30.285151 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:30.285122 2534 patch_prober.go:28] interesting pod/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 12:27:30.285216 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:30.285191 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 12:27:31.284944 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:31.284907 2534 patch_prober.go:28] interesting pod/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 12:27:31.285339 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:31.284965 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 12:27:32.284691 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:32.284659 2534 patch_prober.go:28] interesting pod/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 12:27:32.284893 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:32.284715 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 12:27:33.285174 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:33.285139 2534 patch_prober.go:28] interesting pod/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 12:27:33.285596 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:33.285197 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 12:27:34.287852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:34.287816 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:34.288246 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:34.288123 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:34.288974 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:34.288955 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:27:54.591906 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:54.591871 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7"] Jul 22 12:27:54.592475 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:54.592129 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" containerID="cri-o://b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962" gracePeriod=30 Jul 22 12:27:57.572277 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:57.572244 2534 patch_prober.go:28] interesting pod/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 12:27:57.572677 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:57.572312 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 12:27:59.844697 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.844671 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:27:59.945467 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945432 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-data\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945467 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945470 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-socket\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945490 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-certs\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945531 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-credential-socket\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945550 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-podinfo\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945605 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4v6h\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-kube-api-access-m4v6h\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945668 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b80be94-ac35-4d65-92a9-d836d01f58ed-istiod-ca-cert\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.945726 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945713 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-token\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.946094 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945738 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-envoy\") pod \"0b80be94-ac35-4d65-92a9-d836d01f58ed\" (UID: \"0b80be94-ac35-4d65-92a9-d836d01f58ed\") " Jul 22 12:27:59.946094 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.945769 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:59.946094 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.946058 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:59.946247 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.946187 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:59.947892 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.946686 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b80be94-ac35-4d65-92a9-d836d01f58ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:27:59.947892 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.946742 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:27:59.947892 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.947279 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:59.947892 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.947303 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:59.947892 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.947318 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:27:59.948713 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.948678 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:27:59.949829 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.949223 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-kube-api-access-m4v6h" (OuterVolumeSpecName: "kube-api-access-m4v6h") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "kube-api-access-m4v6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:27:59.950838 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.949930 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:27:59.955290 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:27:59.955268 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b80be94-ac35-4d65-92a9-d836d01f58ed" (UID: "0b80be94-ac35-4d65-92a9-d836d01f58ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:28:00.048004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.047976 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:00.048004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.048003 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4v6h\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-kube-api-access-m4v6h\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:00.048004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.048013 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b80be94-ac35-4d65-92a9-d836d01f58ed-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:00.048237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.048021 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:00.048237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.048029 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:00.048237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.048036 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b80be94-ac35-4d65-92a9-d836d01f58ed-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:00.138865 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.138835 2534 generic.go:356] "Generic (PLEG): container finished" podID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerID="b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962" exitCode=0 Jul 22 12:28:00.139029 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.138914 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" Jul 22 12:28:00.139029 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.138919 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" event={"ID":"0b80be94-ac35-4d65-92a9-d836d01f58ed","Type":"ContainerDied","Data":"b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962"} Jul 22 12:28:00.139104 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.139030 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7" event={"ID":"0b80be94-ac35-4d65-92a9-d836d01f58ed","Type":"ContainerDied","Data":"884bba4b0733245728e07b480a8dc3728eeb6ef5ef22e6f1dc69f614957ac890"} Jul 22 12:28:00.139104 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.139053 2534 scope.go:117] "RemoveContainer" containerID="b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962" Jul 22 12:28:00.151244 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.151222 2534 scope.go:117] "RemoveContainer" containerID="69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8" Jul 22 12:28:00.161191 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.161172 2534 scope.go:117] "RemoveContainer" containerID="b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962" Jul 22 12:28:00.161491 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:28:00.161472 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962\": container with ID starting with b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962 not found: ID does not exist" containerID="b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962" Jul 22 12:28:00.161583 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.161504 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962"} err="failed to get container status \"b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962\": rpc error: code = NotFound desc = could not find container \"b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962\": container with ID starting with b9f9c5e103ac1439d66b7b3afaddb05454cd5f3c045c2afa3c810d25f0444962 not found: ID does not exist" Jul 22 12:28:00.161583 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.161528 2534 scope.go:117] "RemoveContainer" containerID="69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8" Jul 22 12:28:00.161769 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:28:00.161750 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8\": container with ID starting with 69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8 not found: ID does not exist" containerID="69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8" Jul 22 12:28:00.161848 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.161777 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8"} err="failed to get container status \"69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8\": rpc error: code = NotFound desc = could not find container \"69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8\": container with ID starting with 69e32e98ce4f8b05c48f1f8728bce8071e3fe8270a97cc193de187f3ced98aa8 not found: ID does not exist" Jul 22 12:28:00.162028 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.162008 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7"] Jul 22 12:28:00.165422 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:00.165401 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e96939-openshift-default-7979d96c7-ghmk7"] Jul 22 12:28:02.103086 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:02.103048 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" path="/var/lib/kubelet/pods/0b80be94-ac35-4d65-92a9-d836d01f58ed/volumes" Jul 22 12:28:22.051289 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.051252 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv"] Jul 22 12:28:22.051754 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.051513 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" containerID="cri-o://91d4c66b6f8d02184e234b2239f3dc5be21f712496372d5403fa6ef0965416f4" gracePeriod=30 Jul 22 12:28:22.941078 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.940998 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb"] Jul 22 12:28:22.941460 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.941447 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" Jul 22 12:28:22.941504 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.941463 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" Jul 22 12:28:22.941541 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.941536 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" Jul 22 12:28:22.941574 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.941546 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" Jul 22 12:28:22.944762 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.944745 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:22.947268 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.947245 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0b54ae-openshift-default-dockercfg-zfssl\"" Jul 22 12:28:22.955603 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:22.955576 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb"] Jul 22 12:28:23.053812 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.053768 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-certs\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.053849 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-envoy\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.053889 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2wh\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-kube-api-access-qd2wh\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.053913 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istiod-ca-cert\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.053988 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-socket\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.054017 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-data\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.054076 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-credential-socket\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.054115 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-token\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.054197 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.054160 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-podinfo\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.154822 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.154763 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-socket\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.154822 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.154829 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-data\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.154876 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-credential-socket\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.154908 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-token\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.154970 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-podinfo\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155003 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-certs\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155071 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155049 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-envoy\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155078 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2wh\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-kube-api-access-qd2wh\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155108 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istiod-ca-cert\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155212 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-socket\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155489 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155332 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-credential-socket\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155489 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155356 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-data\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155489 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155425 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-certs\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.155821 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.155783 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istiod-ca-cert\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.157428 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.157403 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-envoy\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.157716 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.157690 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-podinfo\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.163870 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.163849 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-token\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.163976 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.163868 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2wh\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-kube-api-access-qd2wh\") pod \"e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.256467 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.256364 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:23.404536 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.404508 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb"] Jul 22 12:28:23.405158 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:28:23.405137 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7cddaaf_4f77_4599_88ed_2bf6a5bf1814.slice/crio-3ac99e8ee5683863d4a2607b04b40deec23689542828860e7ad5133f568a713f WatchSource:0}: Error finding container 3ac99e8ee5683863d4a2607b04b40deec23689542828860e7ad5133f568a713f: Status 404 returned error can't find the container with id 3ac99e8ee5683863d4a2607b04b40deec23689542828860e7ad5133f568a713f Jul 22 12:28:23.407336 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.407298 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:23.407430 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.407365 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:23.407430 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:23.407394 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:24.275237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:24.275204 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" event={"ID":"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814","Type":"ContainerStarted","Data":"4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d"} Jul 22 12:28:24.275732 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:24.275260 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" event={"ID":"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814","Type":"ContainerStarted","Data":"3ac99e8ee5683863d4a2607b04b40deec23689542828860e7ad5133f568a713f"} Jul 22 12:28:24.298883 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:24.298838 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podStartSLOduration=2.298824549 podStartE2EDuration="2.298824549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:28:24.297665859 +0000 UTC m=+2332.623513160" watchObservedRunningTime="2026-07-22 12:28:24.298824549 +0000 UTC m=+2332.624671813" Jul 22 12:28:25.257431 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:25.257395 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:25.263432 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:25.263408 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:25.281162 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:25.281132 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:25.281778 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:25.281761 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:27.295262 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.295229 2534 generic.go:356] "Generic (PLEG): container finished" podID="4460350e-e753-418f-8164-2d7c46f67703" containerID="91d4c66b6f8d02184e234b2239f3dc5be21f712496372d5403fa6ef0965416f4" exitCode=0 Jul 22 12:28:27.295671 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.295315 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" event={"ID":"4460350e-e753-418f-8164-2d7c46f67703","Type":"ContainerDied","Data":"91d4c66b6f8d02184e234b2239f3dc5be21f712496372d5403fa6ef0965416f4"} Jul 22 12:28:27.295671 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.295391 2534 scope.go:117] "RemoveContainer" containerID="6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a" Jul 22 12:28:27.312569 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:28:27.312535 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a\": container with ID starting with 6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a not found: ID does not exist" containerID="6ba373e78d0898761ef29cd98d5d5d24d0077064c2a50f1b8c6fc348f9c6063a" Jul 22 12:28:27.316248 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.316228 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:28:27.394868 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.394840 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-istio-token\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395025 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.394874 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4460350e-e753-418f-8164-2d7c46f67703-istiod-ca-cert\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395025 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.394900 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-credential-socket\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395108 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395090 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-envoy\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395153 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395129 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-certs\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395194 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395171 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prbkc\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-kube-api-access-prbkc\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395239 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395211 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4460350e-e753-418f-8164-2d7c46f67703-istio-podinfo\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395239 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395211 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:28:27.395344 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395274 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-data\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395344 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395320 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-socket\") pod \"4460350e-e753-418f-8164-2d7c46f67703\" (UID: \"4460350e-e753-418f-8164-2d7c46f67703\") " Jul 22 12:28:27.395344 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395270 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4460350e-e753-418f-8164-2d7c46f67703-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:28:27.395695 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395565 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:28:27.395695 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395576 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:28:27.395932 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395767 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4460350e-e753-418f-8164-2d7c46f67703-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.395932 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395789 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.395932 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395840 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.395932 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.395854 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.396366 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.396342 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-data" (OuterVolumeSpecName: "istio-data") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:28:27.397554 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.397533 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4460350e-e753-418f-8164-2d7c46f67703-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:28:27.397722 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.397706 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-istio-token" (OuterVolumeSpecName: "istio-token") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:28:27.398124 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.398107 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-kube-api-access-prbkc" (OuterVolumeSpecName: "kube-api-access-prbkc") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "kube-api-access-prbkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:28:27.398189 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.398129 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4460350e-e753-418f-8164-2d7c46f67703" (UID: "4460350e-e753-418f-8164-2d7c46f67703"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:28:27.497102 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.497074 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.497102 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.497099 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.497288 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.497109 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prbkc\" (UniqueName: \"kubernetes.io/projected/4460350e-e753-418f-8164-2d7c46f67703-kube-api-access-prbkc\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.497288 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.497120 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4460350e-e753-418f-8164-2d7c46f67703-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:27.497288 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:27.497129 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4460350e-e753-418f-8164-2d7c46f67703-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:28:28.302270 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:28.302239 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" event={"ID":"4460350e-e753-418f-8164-2d7c46f67703","Type":"ContainerDied","Data":"64dff689ce94c225d6904ea2a8f6279968b63ed17d3ec2c97f23e69b1a36678e"} Jul 22 12:28:28.302270 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:28.302264 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv" Jul 22 12:28:28.302725 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:28.302296 2534 scope.go:117] "RemoveContainer" containerID="91d4c66b6f8d02184e234b2239f3dc5be21f712496372d5403fa6ef0965416f4" Jul 22 12:28:28.321282 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:28.321253 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv"] Jul 22 12:28:28.324129 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:28.324107 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9edc58-openshift-default-85685c5c8f-5rcxv"] Jul 22 12:28:30.102856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:30.102820 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4460350e-e753-418f-8164-2d7c46f67703" path="/var/lib/kubelet/pods/4460350e-e753-418f-8164-2d7c46f67703/volumes" Jul 22 12:28:31.408281 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408246 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l"] Jul 22 12:28:31.408872 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408856 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" Jul 22 12:28:31.408931 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408876 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" Jul 22 12:28:31.408931 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408894 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" Jul 22 12:28:31.408931 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408913 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" Jul 22 12:28:31.408931 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408929 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" Jul 22 12:28:31.409062 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.408939 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b80be94-ac35-4d65-92a9-d836d01f58ed" containerName="istio-proxy" Jul 22 12:28:31.409062 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.409022 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" Jul 22 12:28:31.413832 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.413811 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.416363 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.416342 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-55573a-openshift-default-dockercfg-hm8fq\"" Jul 22 12:28:31.423739 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.423713 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l"] Jul 22 12:28:31.534161 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534124 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-podinfo\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534161 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534161 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-data\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534410 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534197 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-token\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534410 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534254 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-certs\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534410 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534281 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-envoy\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534410 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534312 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65tbz\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-kube-api-access-65tbz\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534606 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534410 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-socket\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534606 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534470 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istiod-ca-cert\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.534606 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.534502 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-credential-socket\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635348 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635311 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-podinfo\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635348 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635351 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-data\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635382 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-token\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635446 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-certs\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635472 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-envoy\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635509 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65tbz\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-kube-api-access-65tbz\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635530 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-socket\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635564 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istiod-ca-cert\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635942 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635744 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-credential-socket\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635942 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635779 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-data\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635942 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635838 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-certs\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.635942 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.635868 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-socket\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.636081 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.636026 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-credential-socket\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.636315 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.636295 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istiod-ca-cert\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.637935 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.637915 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-envoy\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.638196 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.638181 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-podinfo\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.644088 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.644067 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-token\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.644175 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.644124 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65tbz\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-kube-api-access-65tbz\") pod \"e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.726413 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.726324 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:31.879515 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.879485 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l"] Jul 22 12:28:31.880734 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:28:31.880705 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd1e84e_7ac7_4e00_b7b6_c63e26dbebf9.slice/crio-c35d05ed1dc848b6205d8c6d7c5883eed0c57c9b4ab229e452cd38d761c1a21d WatchSource:0}: Error finding container c35d05ed1dc848b6205d8c6d7c5883eed0c57c9b4ab229e452cd38d761c1a21d: Status 404 returned error can't find the container with id c35d05ed1dc848b6205d8c6d7c5883eed0c57c9b4ab229e452cd38d761c1a21d Jul 22 12:28:31.883096 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.883066 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:31.883149 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.883133 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:31.883196 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:31.883160 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:32.327866 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:32.327835 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" event={"ID":"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9","Type":"ContainerStarted","Data":"2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f"} Jul 22 12:28:32.328044 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:32.327887 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" event={"ID":"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9","Type":"ContainerStarted","Data":"c35d05ed1dc848b6205d8c6d7c5883eed0c57c9b4ab229e452cd38d761c1a21d"} Jul 22 12:28:32.352634 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:32.352579 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podStartSLOduration=1.352562725 podStartE2EDuration="1.352562725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:28:32.351878711 +0000 UTC m=+2340.677725977" watchObservedRunningTime="2026-07-22 12:28:32.352562725 +0000 UTC m=+2340.678409988" Jul 22 12:28:32.570119 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:32.570080 2534 scope.go:117] "RemoveContainer" containerID="14885fb340b600b05a1a465130798712bf16825ffc54644eb9b99b5d72779826" Jul 22 12:28:32.727384 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:32.727346 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:32.732751 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:32.732725 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:33.335881 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:33.335852 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:33.337028 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:33.337009 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:40.392441 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:40.391097 2534 patch_prober.go:28] interesting pod/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": read tcp 10.132.0.2:35100->10.132.0.68:15021: read: connection reset by peer" start-of-body= Jul 22 12:28:40.392441 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:40.391169 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": read tcp 10.132.0.2:35100->10.132.0.68:15021: read: connection reset by peer" Jul 22 12:28:41.386399 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:41.386361 2534 generic.go:356] "Generic (PLEG): container finished" podID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerID="4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d" exitCode=0 Jul 22 12:28:41.386597 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:41.386429 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" event={"ID":"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814","Type":"ContainerDied","Data":"4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d"} Jul 22 12:28:41.386891 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:41.386875 2534 scope.go:117] "RemoveContainer" containerID="4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d" Jul 22 12:28:41.387934 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:41.387899 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:41.388007 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:41.387991 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:41.388053 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:41.388038 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:42.394077 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:42.394034 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" event={"ID":"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814","Type":"ContainerStarted","Data":"36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f"} Jul 22 12:28:43.257588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:43.257550 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:43.258025 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:43.257925 2534 patch_prober.go:28] interesting pod/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 22 12:28:43.258218 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:43.258193 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 22 12:28:44.257917 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:44.257881 2534 patch_prober.go:28] interesting pod/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 22 12:28:44.258539 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:44.258505 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 22 12:28:45.257505 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:45.257468 2534 patch_prober.go:28] interesting pod/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 22 12:28:45.257695 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:45.257555 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 22 12:28:46.256947 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:46.256910 2534 patch_prober.go:28] interesting pod/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 22 12:28:46.257441 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:46.256992 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 22 12:28:47.260388 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:47.260352 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:47.260908 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:47.260891 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:47.261835 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:47.261814 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:28:48.337875 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.337839 2534 patch_prober.go:28] interesting pod/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 22 12:28:48.338328 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.337905 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 22 12:28:48.430482 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.430451 2534 generic.go:356] "Generic (PLEG): container finished" podID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerID="2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f" exitCode=0 Jul 22 12:28:48.430679 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.430505 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" event={"ID":"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9","Type":"ContainerDied","Data":"2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f"} Jul 22 12:28:48.431152 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.431129 2534 scope.go:117] "RemoveContainer" containerID="2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f" Jul 22 12:28:48.432094 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.432065 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:48.432188 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.432168 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:48.432249 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:48.432215 2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:28:49.442500 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:49.442427 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" event={"ID":"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9","Type":"ContainerStarted","Data":"e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9"} Jul 22 12:28:49.727928 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:49.727828 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:49.728965 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:49.728189 2534 patch_prober.go:28] interesting pod/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 22 12:28:49.729175 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:49.729151 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 22 12:28:50.727706 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:50.727667 2534 patch_prober.go:28] interesting pod/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 22 12:28:50.728319 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:50.728290 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 22 12:28:51.727738 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:51.727293 2534 patch_prober.go:28] interesting pod/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 22 12:28:51.727738 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:51.727358 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 22 12:28:52.727531 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:52.727494 2534 patch_prober.go:28] interesting pod/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 22 12:28:52.727838 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:52.727783 2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 22 12:28:53.730808 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:53.730766 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:53.731263 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:53.731246 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:28:53.731826 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:28:53.731784 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:29:28.711384 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.711353 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6"] Jul 22 12:29:28.711807 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.711774 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="4460350e-e753-418f-8164-2d7c46f67703" containerName="istio-proxy" Jul 22 12:29:28.714898 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.714882 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.717729 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.717706 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-55573a\"/\"kube-root-ca.crt\"" Jul 22 12:29:28.718681 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.718664 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-55573a\"/\"auth-test-model-55573a-kserve-self-signed-certs\"" Jul 22 12:29:28.718780 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.718680 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-55573a\"/\"openshift-service-ca.crt\"" Jul 22 12:29:28.718780 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.718680 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-55573a\"/\"default-dockercfg-4ggz6\"" Jul 22 12:29:28.728383 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.728357 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6"] Jul 22 12:29:28.776503 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.776477 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tls-certs\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.776649 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.776517 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-home\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.776649 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.776535 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8v6z\" (UniqueName: \"kubernetes.io/projected/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-kube-api-access-d8v6z\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.776649 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.776563 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-dshm\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.776649 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.776609 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-model-cache\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.776786 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.776658 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tmp-dir\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.877638 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.877599 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tmp-dir\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.877852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.877663 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tls-certs\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.877852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.877706 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-home\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.877852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.877723 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8v6z\" (UniqueName: \"kubernetes.io/projected/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-kube-api-access-d8v6z\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.877852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.877745 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-dshm\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.877852 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.877789 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-model-cache\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.878156 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.878040 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tmp-dir\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.878156 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.878102 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-home\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.878261 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.878199 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-model-cache\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.880086 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.880066 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-dshm\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.880463 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.880442 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tls-certs\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:28.887193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:28.887169 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8v6z\" (UniqueName: \"kubernetes.io/projected/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-kube-api-access-d8v6z\") pod \"auth-test-model-55573a-kserve-9c94c5d75-bzfv6\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:29.024654 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:29.024565 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:29.167668 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:29.167646 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6"] Jul 22 12:29:29.169901 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:29:29.169865 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81a2345b_1f77_41a5_bd67_49ebffd3ac2e.slice/crio-657b4a3b3472d62ff81cfe92573a267dd7be35a38dfcdeebd273b343858eb18b WatchSource:0}: Error finding container 657b4a3b3472d62ff81cfe92573a267dd7be35a38dfcdeebd273b343858eb18b: Status 404 returned error can't find the container with id 657b4a3b3472d62ff81cfe92573a267dd7be35a38dfcdeebd273b343858eb18b Jul 22 12:29:29.676716 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:29.676682 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" event={"ID":"81a2345b-1f77-41a5-bd67-49ebffd3ac2e","Type":"ContainerStarted","Data":"4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76"} Jul 22 12:29:29.676911 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:29.676725 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" event={"ID":"81a2345b-1f77-41a5-bd67-49ebffd3ac2e","Type":"ContainerStarted","Data":"657b4a3b3472d62ff81cfe92573a267dd7be35a38dfcdeebd273b343858eb18b"} Jul 22 12:29:29.694776 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:29.694735 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" podStartSLOduration=1.694722084 podStartE2EDuration="1.694722084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:29:29.69386502 +0000 UTC m=+2398.019712285" watchObservedRunningTime="2026-07-22 12:29:29.694722084 +0000 UTC m=+2398.020569345" Jul 22 12:29:32.316463 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:32.316420 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:29:32.321410 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:32.321380 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:29:39.025265 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:39.025225 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:39.025784 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:39.025284 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:39.037625 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:39.037602 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:29:39.741054 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:29:39.741020 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:31:29.541261 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:29.541186 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6"] Jul 22 12:31:44.562243 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.562179 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" podUID="81a2345b-1f77-41a5-bd67-49ebffd3ac2e" containerName="main" containerID="cri-o://4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76" gracePeriod=45 Jul 22 12:31:44.814189 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.814125 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:31:44.879182 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879150 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8v6z\" (UniqueName: \"kubernetes.io/projected/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-kube-api-access-d8v6z\") pod \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " Jul 22 12:31:44.879373 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879237 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tls-certs\") pod \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " Jul 22 12:31:44.879373 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879259 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tmp-dir\") pod \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " Jul 22 12:31:44.879373 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879291 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-model-cache\") pod \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " Jul 22 12:31:44.879373 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879366 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-dshm\") pod \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " Jul 22 12:31:44.879612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879412 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-home\") pod \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\" (UID: \"81a2345b-1f77-41a5-bd67-49ebffd3ac2e\") " Jul 22 12:31:44.879668 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879601 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81a2345b-1f77-41a5-bd67-49ebffd3ac2e" (UID: "81a2345b-1f77-41a5-bd67-49ebffd3ac2e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:31:44.879668 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879613 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-model-cache" (OuterVolumeSpecName: "model-cache") pod "81a2345b-1f77-41a5-bd67-49ebffd3ac2e" (UID: "81a2345b-1f77-41a5-bd67-49ebffd3ac2e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:31:44.879787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.879763 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-home" (OuterVolumeSpecName: "home") pod "81a2345b-1f77-41a5-bd67-49ebffd3ac2e" (UID: "81a2345b-1f77-41a5-bd67-49ebffd3ac2e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:31:44.881690 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.881665 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81a2345b-1f77-41a5-bd67-49ebffd3ac2e" (UID: "81a2345b-1f77-41a5-bd67-49ebffd3ac2e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 12:31:44.881784 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.881690 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-kube-api-access-d8v6z" (OuterVolumeSpecName: "kube-api-access-d8v6z") pod "81a2345b-1f77-41a5-bd67-49ebffd3ac2e" (UID: "81a2345b-1f77-41a5-bd67-49ebffd3ac2e"). InnerVolumeSpecName "kube-api-access-d8v6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:31:44.881784 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.881735 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-dshm" (OuterVolumeSpecName: "dshm") pod "81a2345b-1f77-41a5-bd67-49ebffd3ac2e" (UID: "81a2345b-1f77-41a5-bd67-49ebffd3ac2e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:31:44.980293 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.980259 2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-home\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:31:44.980293 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.980289 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8v6z\" (UniqueName: \"kubernetes.io/projected/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-kube-api-access-d8v6z\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:31:44.980293 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.980300 2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tls-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:31:44.980527 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.980309 2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-tmp-dir\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:31:44.980527 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.980317 2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-model-cache\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:31:44.980527 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:44.980324 2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81a2345b-1f77-41a5-bd67-49ebffd3ac2e-dshm\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:31:45.447101 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.447065 2534 generic.go:356] "Generic (PLEG): container finished" podID="81a2345b-1f77-41a5-bd67-49ebffd3ac2e" containerID="4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76" exitCode=0 Jul 22 12:31:45.447295 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.447113 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" event={"ID":"81a2345b-1f77-41a5-bd67-49ebffd3ac2e","Type":"ContainerDied","Data":"4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76"} Jul 22 12:31:45.447295 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.447131 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" Jul 22 12:31:45.447295 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.447150 2534 scope.go:117] "RemoveContainer" containerID="4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76" Jul 22 12:31:45.447295 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.447139 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6" event={"ID":"81a2345b-1f77-41a5-bd67-49ebffd3ac2e","Type":"ContainerDied","Data":"657b4a3b3472d62ff81cfe92573a267dd7be35a38dfcdeebd273b343858eb18b"} Jul 22 12:31:45.459145 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.459116 2534 scope.go:117] "RemoveContainer" containerID="4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76" Jul 22 12:31:45.459618 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:31:45.459590 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76\": container with ID starting with 4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76 not found: ID does not exist" containerID="4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76" Jul 22 12:31:45.459712 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.459630 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76"} err="failed to get container status \"4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76\": rpc error: code = NotFound desc = could not find container \"4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76\": container with ID starting with 4b91d82fbe3318b843b85fa5eca1fee7c1546aba9dc87fc153809901c7a6fc76 not found: ID does not exist" Jul 22 12:31:45.480342 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.480309 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6"] Jul 22 12:31:45.487846 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:45.487821 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-55573a/auth-test-model-55573a-kserve-9c94c5d75-bzfv6"] Jul 22 12:31:46.102058 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:31:46.102024 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a2345b-1f77-41a5-bd67-49ebffd3ac2e" path="/var/lib/kubelet/pods/81a2345b-1f77-41a5-bd67-49ebffd3ac2e/volumes" Jul 22 12:32:49.209911 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.209870 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb"] Jul 22 12:32:49.210351 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.210243 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a2345b-1f77-41a5-bd67-49ebffd3ac2e" containerName="main" Jul 22 12:32:49.210351 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.210253 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a2345b-1f77-41a5-bd67-49ebffd3ac2e" containerName="main" Jul 22 12:32:49.210351 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.210313 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a2345b-1f77-41a5-bd67-49ebffd3ac2e" containerName="main" Jul 22 12:32:49.213464 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.213427 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.219742 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.219722 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b54ae\"/\"kube-root-ca.crt\"" Jul 22 12:32:49.220921 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.220892 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0b54ae\"/\"openshift-service-ca.crt\"" Jul 22 12:32:49.221025 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.220898 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b54ae\"/\"rate-test-model-0b54ae-kserve-self-signed-certs\"" Jul 22 12:32:49.221025 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.220942 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0b54ae\"/\"default-dockercfg-t29dw\"" Jul 22 12:32:49.246104 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.246082 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb"] Jul 22 12:32:49.314783 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.314751 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-dshm\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.314783 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.314813 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-tmp-dir\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.315000 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.314890 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-model-cache\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.315000 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.314945 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwl52\" (UniqueName: \"kubernetes.io/projected/6f8da97b-77de-4770-b202-e9067e29674f-kube-api-access-bwl52\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.315000 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.314996 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8da97b-77de-4770-b202-e9067e29674f-tls-certs\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.315114 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.315019 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-home\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.415973 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.415937 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-model-cache\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416168 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.415990 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwl52\" (UniqueName: \"kubernetes.io/projected/6f8da97b-77de-4770-b202-e9067e29674f-kube-api-access-bwl52\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416168 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416036 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8da97b-77de-4770-b202-e9067e29674f-tls-certs\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416168 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416066 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-home\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416168 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416119 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-dshm\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416410 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416379 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-tmp-dir\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416475 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416459 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-home\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416535 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416378 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-model-cache\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.416609 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.416585 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-tmp-dir\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.418550 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.418529 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-dshm\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.418961 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.418933 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8da97b-77de-4770-b202-e9067e29674f-tls-certs\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.428302 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.428283 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwl52\" (UniqueName: \"kubernetes.io/projected/6f8da97b-77de-4770-b202-e9067e29674f-kube-api-access-bwl52\") pod \"rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.522944 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.522871 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:49.673324 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.673299 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb"] Jul 22 12:32:49.674554 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:32:49.674523 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f8da97b_77de_4770_b202_e9067e29674f.slice/crio-0225a5e63924d7383b61a6c53f15cb02e8f1f1d99df402f8de4ed71977b9e12e WatchSource:0}: Error finding container 0225a5e63924d7383b61a6c53f15cb02e8f1f1d99df402f8de4ed71977b9e12e: Status 404 returned error can't find the container with id 0225a5e63924d7383b61a6c53f15cb02e8f1f1d99df402f8de4ed71977b9e12e Jul 22 12:32:49.821401 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.821313 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" event={"ID":"6f8da97b-77de-4770-b202-e9067e29674f","Type":"ContainerStarted","Data":"6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f"} Jul 22 12:32:49.821401 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.821361 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" event={"ID":"6f8da97b-77de-4770-b202-e9067e29674f","Type":"ContainerStarted","Data":"0225a5e63924d7383b61a6c53f15cb02e8f1f1d99df402f8de4ed71977b9e12e"} Jul 22 12:32:49.854915 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:49.854851 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" podStartSLOduration=0.854836607 podStartE2EDuration="854.836607ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:32:49.854346774 +0000 UTC m=+2598.180194039" watchObservedRunningTime="2026-07-22 12:32:49.854836607 +0000 UTC m=+2598.180683867" Jul 22 12:32:59.523971 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:59.523872 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:59.523971 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:59.523940 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:59.536735 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:59.536705 2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:32:59.889959 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:32:59.889931 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:33:58.908880 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:33:58.908846 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb"] Jul 22 12:34:13.929502 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:13.929443 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" podUID="6f8da97b-77de-4770-b202-e9067e29674f" containerName="main" containerID="cri-o://6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f" gracePeriod=45 Jul 22 12:34:14.188000 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.187942 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:34:14.215846 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.215819 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-tmp-dir\") pod \"6f8da97b-77de-4770-b202-e9067e29674f\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " Jul 22 12:34:14.216004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.215856 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwl52\" (UniqueName: \"kubernetes.io/projected/6f8da97b-77de-4770-b202-e9067e29674f-kube-api-access-bwl52\") pod \"6f8da97b-77de-4770-b202-e9067e29674f\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " Jul 22 12:34:14.216004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.215892 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-home\") pod \"6f8da97b-77de-4770-b202-e9067e29674f\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " Jul 22 12:34:14.216004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.215930 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8da97b-77de-4770-b202-e9067e29674f-tls-certs\") pod \"6f8da97b-77de-4770-b202-e9067e29674f\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " Jul 22 12:34:14.216165 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.216013 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-model-cache\") pod \"6f8da97b-77de-4770-b202-e9067e29674f\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " Jul 22 12:34:14.216165 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.216117 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-dshm\") pod \"6f8da97b-77de-4770-b202-e9067e29674f\" (UID: \"6f8da97b-77de-4770-b202-e9067e29674f\") " Jul 22 12:34:14.217304 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.216901 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-home" (OuterVolumeSpecName: "home") pod "6f8da97b-77de-4770-b202-e9067e29674f" (UID: "6f8da97b-77de-4770-b202-e9067e29674f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:14.217466 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.217303 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f8da97b-77de-4770-b202-e9067e29674f" (UID: "6f8da97b-77de-4770-b202-e9067e29674f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:14.218439 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.218288 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f8da97b-77de-4770-b202-e9067e29674f" (UID: "6f8da97b-77de-4770-b202-e9067e29674f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:14.219308 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.219282 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-dshm" (OuterVolumeSpecName: "dshm") pod "6f8da97b-77de-4770-b202-e9067e29674f" (UID: "6f8da97b-77de-4770-b202-e9067e29674f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:14.219858 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.219836 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f8da97b-77de-4770-b202-e9067e29674f-kube-api-access-bwl52" (OuterVolumeSpecName: "kube-api-access-bwl52") pod "6f8da97b-77de-4770-b202-e9067e29674f" (UID: "6f8da97b-77de-4770-b202-e9067e29674f"). InnerVolumeSpecName "kube-api-access-bwl52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:34:14.219982 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.219960 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f8da97b-77de-4770-b202-e9067e29674f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f8da97b-77de-4770-b202-e9067e29674f" (UID: "6f8da97b-77de-4770-b202-e9067e29674f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 12:34:14.299257 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.299224 2534 generic.go:356] "Generic (PLEG): container finished" podID="6f8da97b-77de-4770-b202-e9067e29674f" containerID="6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f" exitCode=0 Jul 22 12:34:14.299447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.299268 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" event={"ID":"6f8da97b-77de-4770-b202-e9067e29674f","Type":"ContainerDied","Data":"6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f"} Jul 22 12:34:14.299447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.299273 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" Jul 22 12:34:14.299447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.299299 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb" event={"ID":"6f8da97b-77de-4770-b202-e9067e29674f","Type":"ContainerDied","Data":"0225a5e63924d7383b61a6c53f15cb02e8f1f1d99df402f8de4ed71977b9e12e"} Jul 22 12:34:14.299447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.299319 2534 scope.go:117] "RemoveContainer" containerID="6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f" Jul 22 12:34:14.308880 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.308863 2534 scope.go:117] "RemoveContainer" containerID="6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f" Jul 22 12:34:14.309158 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:34:14.309140 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f\": container with ID starting with 6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f not found: ID does not exist" containerID="6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f" Jul 22 12:34:14.309220 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.309169 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f"} err="failed to get container status \"6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f\": rpc error: code = NotFound desc = could not find container \"6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f\": container with ID starting with 6b8ae5e14a56d384469d750e5be9610b35e4b740ce32d7b3bfa840fabd20b60f not found: ID does not exist" Jul 22 12:34:14.317127 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.317104 2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-dshm\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:14.317127 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.317127 2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-tmp-dir\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:14.317287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.317139 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwl52\" (UniqueName: \"kubernetes.io/projected/6f8da97b-77de-4770-b202-e9067e29674f-kube-api-access-bwl52\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:14.317287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.317151 2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-home\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:14.317287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.317164 2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8da97b-77de-4770-b202-e9067e29674f-tls-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:14.317287 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.317174 2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8da97b-77de-4770-b202-e9067e29674f-model-cache\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:14.323385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.323360 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb"] Jul 22 12:34:14.330550 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:14.330519 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0b54ae/rate-test-model-0b54ae-kserve-6f7c45764c-hf2jb"] Jul 22 12:34:16.102154 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:16.102123 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f8da97b-77de-4770-b202-e9067e29674f" path="/var/lib/kubelet/pods/6f8da97b-77de-4770-b202-e9067e29674f/volumes" Jul 22 12:34:32.356095 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:32.356021 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:34:32.363067 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:32.363042 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:34:41.514671 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:41.514629 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb"] Jul 22 12:34:41.515074 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:41.514914 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" containerID="cri-o://36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f" gracePeriod=30 Jul 22 12:34:46.772095 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.772070 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:34:46.917289 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917262 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-socket\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917456 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917299 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-data\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917456 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917320 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-credential-socket\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917456 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917359 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-podinfo\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917456 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917387 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istiod-ca-cert\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917456 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917426 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-certs\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917696 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917533 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:46.917696 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917587 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-envoy\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917696 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917657 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-token\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917696 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917683 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd2wh\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-kube-api-access-qd2wh\") pod \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\" (UID: \"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814\") " Jul 22 12:34:46.917937 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917690 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:46.917937 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.917859 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:34:46.918127 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.918092 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:46.918127 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.918099 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:46.918457 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.918144 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:46.918457 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.918160 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:46.918457 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.918166 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:46.919918 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.919896 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:34:46.920011 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.919988 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-kube-api-access-qd2wh" (OuterVolumeSpecName: "kube-api-access-qd2wh") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "kube-api-access-qd2wh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:34:46.920070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.920025 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:34:46.920070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:46.920055 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" (UID: "d7cddaaf-4f77-4599-88ed-2bf6a5bf1814"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:34:47.019016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.018986 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:47.019016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.019010 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:47.019016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.019018 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:47.019223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.019026 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd2wh\" (UniqueName: \"kubernetes.io/projected/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-kube-api-access-qd2wh\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:47.019223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.019034 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:47.019223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.019042 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:34:47.486990 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.486955 2534 generic.go:356] "Generic (PLEG): container finished" podID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerID="36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f" exitCode=0 Jul 22 12:34:47.487193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.487007 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" Jul 22 12:34:47.487193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.487027 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" event={"ID":"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814","Type":"ContainerDied","Data":"36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f"} Jul 22 12:34:47.487193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.487079 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb" event={"ID":"d7cddaaf-4f77-4599-88ed-2bf6a5bf1814","Type":"ContainerDied","Data":"3ac99e8ee5683863d4a2607b04b40deec23689542828860e7ad5133f568a713f"} Jul 22 12:34:47.487193 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.487107 2534 scope.go:117] "RemoveContainer" containerID="36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f" Jul 22 12:34:47.496973 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.496945 2534 scope.go:117] "RemoveContainer" containerID="4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d" Jul 22 12:34:47.505297 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.505280 2534 scope.go:117] "RemoveContainer" containerID="36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f" Jul 22 12:34:47.505533 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:34:47.505517 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f\": container with ID starting with 36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f not found: ID does not exist" containerID="36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f" Jul 22 12:34:47.505588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.505543 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f"} err="failed to get container status \"36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f\": rpc error: code = NotFound desc = could not find container \"36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f\": container with ID starting with 36f81b86968b65b1f0dafc6972b31b20620825c7b3b651e8db37f98cdb741b6f not found: ID does not exist" Jul 22 12:34:47.505588 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.505559 2534 scope.go:117] "RemoveContainer" containerID="4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d" Jul 22 12:34:47.505814 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:34:47.505782 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d\": container with ID starting with 4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d not found: ID does not exist" containerID="4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d" Jul 22 12:34:47.505869 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.505817 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d"} err="failed to get container status \"4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d\": rpc error: code = NotFound desc = could not find container \"4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d\": container with ID starting with 4ee1a86cafab016b60d65041ccc3056d674c9aa41111e4148e961f801c7c988d not found: ID does not exist" Jul 22 12:34:47.518427 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.518405 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb"] Jul 22 12:34:47.526258 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:47.526237 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0b54ae-openshift-default-65c7559f47-j76wb"] Jul 22 12:34:48.101779 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:34:48.101743 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" path="/var/lib/kubelet/pods/d7cddaaf-4f77-4599-88ed-2bf6a5bf1814/volumes" Jul 22 12:35:04.554363 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:04.554327 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l"] Jul 22 12:35:04.580702 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:04.580637 2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" containerID="cri-o://e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9" gracePeriod=30 Jul 22 12:35:05.339385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:05.339341 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c86df85c-snhc7_0ecbc3ef-e051-4e60-9240-61d79341b8f6/authorino/0.log" Jul 22 12:35:08.733612 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:08.733570 2534 patch_prober.go:28] interesting pod/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 12:35:08.734116 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:08.733674 2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 12:35:09.857842 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.857813 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:35:09.929599 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929565 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-credential-socket\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929599 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929606 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-socket\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929631 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-token\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929702 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istiod-ca-cert\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929729 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65tbz\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-kube-api-access-65tbz\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929749 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-data\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929827 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-certs\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.929856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929855 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-envoy\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.930185 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929881 2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-podinfo\") pod \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\" (UID: \"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9\") " Jul 22 12:35:09.930185 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929896 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:35:09.930185 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.929964 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:35:09.930185 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930087 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:35:09.930185 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930170 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:35:09.930435 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930331 2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istiod-ca-cert\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:09.930435 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930349 2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-certs\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:09.930435 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930360 2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-credential-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:09.930435 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930372 2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-workload-socket\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:09.930677 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.930652 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-data" (OuterVolumeSpecName: "istio-data") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:35:09.932043 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.932019 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-kube-api-access-65tbz" (OuterVolumeSpecName: "kube-api-access-65tbz") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "kube-api-access-65tbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:35:09.932175 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.932155 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-token" (OuterVolumeSpecName: "istio-token") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:35:09.932443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.932416 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:35:09.932443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:09.932427 2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" (UID: "5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:35:10.031370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.031334 2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-token\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:10.031370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.031362 2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65tbz\" (UniqueName: \"kubernetes.io/projected/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-kube-api-access-65tbz\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:10.031370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.031372 2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-data\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:10.031370 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.031379 2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-envoy\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:10.031630 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.031387 2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9-istio-podinfo\") on node \"ip-10-0-143-104.ec2.internal\" DevicePath \"\"" Jul 22 12:35:10.614846 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.614783 2534 generic.go:356] "Generic (PLEG): container finished" podID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerID="e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9" exitCode=0 Jul 22 12:35:10.615016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.614857 2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" Jul 22 12:35:10.615016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.614820 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" event={"ID":"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9","Type":"ContainerDied","Data":"e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9"} Jul 22 12:35:10.615016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.614941 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l" event={"ID":"5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9","Type":"ContainerDied","Data":"c35d05ed1dc848b6205d8c6d7c5883eed0c57c9b4ab229e452cd38d761c1a21d"} Jul 22 12:35:10.615016 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.614962 2534 scope.go:117] "RemoveContainer" containerID="e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9" Jul 22 12:35:10.624325 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.624256 2534 scope.go:117] "RemoveContainer" containerID="2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f" Jul 22 12:35:10.632536 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.632515 2534 scope.go:117] "RemoveContainer" containerID="e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9" Jul 22 12:35:10.632780 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:35:10.632761 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9\": container with ID starting with e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9 not found: ID does not exist" containerID="e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9" Jul 22 12:35:10.632856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.632788 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9"} err="failed to get container status \"e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9\": rpc error: code = NotFound desc = could not find container \"e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9\": container with ID starting with e268db5a76c0c8175f146d04657639e4efda007f50d0a3e90a9f86527c73fdd9 not found: ID does not exist" Jul 22 12:35:10.632856 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.632827 2534 scope.go:117] "RemoveContainer" containerID="2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f" Jul 22 12:35:10.633088 ip-10-0-143-104 kubenswrapper[2534]: E0722 12:35:10.633070 2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f\": container with ID starting with 2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f not found: ID does not exist" containerID="2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f" Jul 22 12:35:10.633132 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.633095 2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f"} err="failed to get container status \"2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f\": rpc error: code = NotFound desc = could not find container \"2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f\": container with ID starting with 2048f41daca692ef36195c56fb92998ca8f54ff181e0ef986cd4c31985dcc26f not found: ID does not exist" Jul 22 12:35:10.638033 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.638012 2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l"] Jul 22 12:35:10.643947 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:10.643922 2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-55573a-openshift-default-6b876dbdcf-29r7l"] Jul 22 12:35:11.985700 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:11.985671 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c4d6d56dd-cvj4c_779ecd9c-4776-4fd1-becc-dbf7127314b4/copy-manifests/0.log" Jul 22 12:35:11.993555 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:11.993526 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c4d6d56dd-cvj4c_779ecd9c-4776-4fd1-becc-dbf7127314b4/manager/0.log" Jul 22 12:35:12.102651 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:12.102619 2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" path="/var/lib/kubelet/pods/5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9/volumes" Jul 22 12:35:12.752743 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:12.752710 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745360-7p9r5_492cf9bd-4bdf-4f2b-9c34-93ddd9e6e360/cleanup/0.log" Jul 22 12:35:13.930652 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:13.930621 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx_1a284ee5-e8f7-4005-8a34-80ff91f35254/extract/0.log" Jul 22 12:35:13.941134 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:13.941111 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx_1a284ee5-e8f7-4005-8a34-80ff91f35254/util/0.log" Jul 22 12:35:13.950362 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:13.950343 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx_1a284ee5-e8f7-4005-8a34-80ff91f35254/pull/0.log" Jul 22 12:35:14.088162 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.088129 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv_82fd35dc-c6ab-490e-83c3-c8af34644bf4/util/0.log" Jul 22 12:35:14.098067 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.098036 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv_82fd35dc-c6ab-490e-83c3-c8af34644bf4/pull/0.log" Jul 22 12:35:14.107448 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.107428 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv_82fd35dc-c6ab-490e-83c3-c8af34644bf4/extract/0.log" Jul 22 12:35:14.232292 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.232220 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr_492a79d2-1733-4f8b-8d7e-0be3ce80c189/util/0.log" Jul 22 12:35:14.243066 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.243044 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr_492a79d2-1733-4f8b-8d7e-0be3ce80c189/pull/0.log" Jul 22 12:35:14.252535 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.252515 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr_492a79d2-1733-4f8b-8d7e-0be3ce80c189/extract/0.log" Jul 22 12:35:14.369451 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.369419 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz_6e0389f7-2c3d-41d4-a843-ee045b4dbd87/util/0.log" Jul 22 12:35:14.378142 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.378110 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz_6e0389f7-2c3d-41d4-a843-ee045b4dbd87/pull/0.log" Jul 22 12:35:14.389723 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.389696 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz_6e0389f7-2c3d-41d4-a843-ee045b4dbd87/extract/0.log" Jul 22 12:35:14.518926 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:14.518844 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c86df85c-snhc7_0ecbc3ef-e051-4e60-9240-61d79341b8f6/authorino/0.log" Jul 22 12:35:15.024630 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:15.024598 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-w7g4n_5610964c-edf1-4d86-9e6f-5de894942733/registry-server/0.log" Jul 22 12:35:15.151743 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:15.151707 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tbjv5_8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb/manager/0.log" Jul 22 12:35:15.397330 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:15.397305 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wg5xx_a802b12d-7bee-4ddc-b034-5dc83866d9a4/manager/0.log" Jul 22 12:35:15.792470 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:15.792395 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47p247_54d19681-f9d1-42bd-b173-04f59eea7835/istio-proxy/0.log" Jul 22 12:35:16.289813 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:16.289755 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/1.log" Jul 22 12:35:16.683833 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:16.683769 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79ccd9f656-4kgpj_8b58a72c-7ded-4849-a1fb-df32c70b6081/router/0.log" Jul 22 12:35:17.085014 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:17.084918 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-hcqzb_13f7a223-a224-43d3-a231-7a210b93e7b1/main/0.log" Jul 22 12:35:17.339124 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:17.339053 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-cgb72_06650cb7-9792-4a57-89e8-42421554f281/main/0.log" Jul 22 12:35:17.721706 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:17.721675 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-txn8c_9ea5123a-b713-4493-a810-81138edabfa8/main/0.log" Jul 22 12:35:23.469424 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469386 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mtjsz/must-gather-jtmww"] Jul 22 12:35:23.469884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469838 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" Jul 22 12:35:23.469884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469853 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" Jul 22 12:35:23.469884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469863 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f8da97b-77de-4770-b202-e9067e29674f" containerName="main" Jul 22 12:35:23.469884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469868 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f8da97b-77de-4770-b202-e9067e29674f" containerName="main" Jul 22 12:35:23.469884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469875 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" Jul 22 12:35:23.469884 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469880 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469893 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469898 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469904 2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469909 2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469973 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469981 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469987 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f8da97b-77de-4770-b202-e9067e29674f" containerName="main" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469994 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7cddaaf-4f77-4599-88ed-2bf6a5bf1814" containerName="istio-proxy" Jul 22 12:35:23.470117 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.469999 2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cd1e84e-7ac7-4e00-b7b6-c63e26dbebf9" containerName="istio-proxy" Jul 22 12:35:23.474443 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.474423 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.477168 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.477144 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mtjsz\"/\"kube-root-ca.crt\"" Jul 22 12:35:23.477274 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.477196 2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mtjsz\"/\"default-dockercfg-p49nt\"" Jul 22 12:35:23.478076 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.478059 2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mtjsz\"/\"openshift-service-ca.crt\"" Jul 22 12:35:23.482208 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.482185 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mtjsz/must-gather-jtmww"] Jul 22 12:35:23.557445 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.557416 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htfcs\" (UniqueName: \"kubernetes.io/projected/e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e-kube-api-access-htfcs\") pod \"must-gather-jtmww\" (UID: \"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e\") " pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.557602 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.557480 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e-must-gather-output\") pod \"must-gather-jtmww\" (UID: \"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e\") " pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.658708 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.658678 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e-must-gather-output\") pod \"must-gather-jtmww\" (UID: \"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e\") " pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.658917 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.658747 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htfcs\" (UniqueName: \"kubernetes.io/projected/e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e-kube-api-access-htfcs\") pod \"must-gather-jtmww\" (UID: \"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e\") " pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.659056 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.659037 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e-must-gather-output\") pod \"must-gather-jtmww\" (UID: \"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e\") " pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.668975 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.668955 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htfcs\" (UniqueName: \"kubernetes.io/projected/e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e-kube-api-access-htfcs\") pod \"must-gather-jtmww\" (UID: \"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e\") " pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.784237 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.784148 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mtjsz/must-gather-jtmww" Jul 22 12:35:23.931408 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.931382 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mtjsz/must-gather-jtmww"] Jul 22 12:35:23.932788 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:35:23.932766 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0190b4d_e1f2_4d66_a0bd_00aa1ac2d55e.slice/crio-540bb38747e247a76fa5136fac1a6675276ca6637bb43b8d5ea4f58bccd783df WatchSource:0}: Error finding container 540bb38747e247a76fa5136fac1a6675276ca6637bb43b8d5ea4f58bccd783df: Status 404 returned error can't find the container with id 540bb38747e247a76fa5136fac1a6675276ca6637bb43b8d5ea4f58bccd783df Jul 22 12:35:23.935102 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:23.935083 2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 12:35:24.699458 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:24.699420 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtjsz/must-gather-jtmww" event={"ID":"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e","Type":"ContainerStarted","Data":"540bb38747e247a76fa5136fac1a6675276ca6637bb43b8d5ea4f58bccd783df"} Jul 22 12:35:26.716122 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:26.716089 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtjsz/must-gather-jtmww" event={"ID":"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e","Type":"ContainerStarted","Data":"26fd88c2675c87b2f1df35999714a8c51c8abe3f9bdc88a575a9a81d9788dbe4"} Jul 22 12:35:26.716122 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:26.716130 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtjsz/must-gather-jtmww" event={"ID":"e0190b4d-e1f2-4d66-a0bd-00aa1ac2d55e","Type":"ContainerStarted","Data":"dd41a9d88244107b20465999ea2dc000e718b52a1ae168ebc5215e79654d02ba"} Jul 22 12:35:26.752226 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:26.752172 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mtjsz/must-gather-jtmww" podStartSLOduration=1.907397107 podStartE2EDuration="3.752156065s" totalImagesPullingTime="1.844758958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:35:26.748841083 +0000 UTC m=+2755.074688345" watchObservedRunningTime="2026-07-22 12:35:26.752156065 +0000 UTC m=+2755.078003328" Jul 22 12:35:27.450735 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:27.450699 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dkgxt_ea5ae2be-b7bd-4758-a311-9104121792f1/global-pull-secret-syncer/0.log" Jul 22 12:35:27.721662 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:27.721583 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b8zxh_83528e59-ca1b-4ae2-b520-6fd11121829e/konnectivity-agent/0.log" Jul 22 12:35:27.906361 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:27.906333 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-104.ec2.internal_a1252a5c274066b1c97fa5a755cd7d35/haproxy/0.log" Jul 22 12:35:32.637447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.637411 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx_1a284ee5-e8f7-4005-8a34-80ff91f35254/extract/0.log" Jul 22 12:35:32.687815 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.687405 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx_1a284ee5-e8f7-4005-8a34-80ff91f35254/util/0.log" Jul 22 12:35:32.735503 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.735470 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jrlmx_1a284ee5-e8f7-4005-8a34-80ff91f35254/pull/0.log" Jul 22 12:35:32.803036 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.803004 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv_82fd35dc-c6ab-490e-83c3-c8af34644bf4/extract/0.log" Jul 22 12:35:32.855787 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.855754 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv_82fd35dc-c6ab-490e-83c3-c8af34644bf4/util/0.log" Jul 22 12:35:32.932354 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.932103 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0brcmv_82fd35dc-c6ab-490e-83c3-c8af34644bf4/pull/0.log" Jul 22 12:35:32.995916 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:32.995888 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr_492a79d2-1733-4f8b-8d7e-0be3ce80c189/extract/0.log" Jul 22 12:35:33.046083 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.046057 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr_492a79d2-1733-4f8b-8d7e-0be3ce80c189/util/0.log" Jul 22 12:35:33.100718 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.100691 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bhdlr_492a79d2-1733-4f8b-8d7e-0be3ce80c189/pull/0.log" Jul 22 12:35:33.143325 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.143293 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz_6e0389f7-2c3d-41d4-a843-ee045b4dbd87/extract/0.log" Jul 22 12:35:33.208239 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.208156 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz_6e0389f7-2c3d-41d4-a843-ee045b4dbd87/util/0.log" Jul 22 12:35:33.256305 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.256262 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef157chz_6e0389f7-2c3d-41d4-a843-ee045b4dbd87/pull/0.log" Jul 22 12:35:33.738819 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.735492 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c86df85c-snhc7_0ecbc3ef-e051-4e60-9240-61d79341b8f6/authorino/0.log" Jul 22 12:35:33.937980 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:33.937930 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-w7g4n_5610964c-edf1-4d86-9e6f-5de894942733/registry-server/0.log" Jul 22 12:35:34.530132 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:34.530096 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tbjv5_8bacb1a4-576f-4da2-ad7b-3b7ca836c2eb/manager/0.log" Jul 22 12:35:34.811819 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:34.811755 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wg5xx_a802b12d-7bee-4ddc-b034-5dc83866d9a4/manager/0.log" Jul 22 12:35:36.277886 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:36.277853 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-brh7n_ba13c544-b6f5-4b80-92c8-3f672a89de5e/cluster-monitoring-operator/0.log" Jul 22 12:35:36.434221 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:36.434196 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gms99_abd5a4a0-46ee-441c-98fe-97dc1e00c6d2/node-exporter/0.log" Jul 22 12:35:36.457960 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:36.457929 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gms99_abd5a4a0-46ee-441c-98fe-97dc1e00c6d2/kube-rbac-proxy/0.log" Jul 22 12:35:36.496627 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:36.496603 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gms99_abd5a4a0-46ee-441c-98fe-97dc1e00c6d2/init-textfile/0.log" Jul 22 12:35:37.038739 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.038692 2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d"] Jul 22 12:35:37.041358 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.041325 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-k4lxd_878134b9-f138-4d1f-96cb-ad899a981140/prometheus-operator-admission-webhook/0.log" Jul 22 12:35:37.047057 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.047033 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.050923 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.050896 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d"] Jul 22 12:35:37.083011 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.082985 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-644d6f7956-dmd4d_154a039d-250f-4a66-a6b2-9f7148424352/telemeter-client/0.log" Jul 22 12:35:37.113322 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.113294 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-644d6f7956-dmd4d_154a039d-250f-4a66-a6b2-9f7148424352/reload/0.log" Jul 22 12:35:37.138302 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.138273 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-644d6f7956-dmd4d_154a039d-250f-4a66-a6b2-9f7148424352/kube-rbac-proxy/0.log" Jul 22 12:35:37.228857 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.228822 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-podres\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.229084 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.228882 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-proc\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.229084 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.228988 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-lib-modules\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.229084 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.229058 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqc4f\" (UniqueName: \"kubernetes.io/projected/2f8084ad-f30a-4343-8696-1ce5755a1c82-kube-api-access-sqc4f\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.229248 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.229122 2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-sys\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.330859 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.330750 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-podres\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.330859 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.330847 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-proc\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.330886 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-lib-modules\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.330945 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqc4f\" (UniqueName: \"kubernetes.io/projected/2f8084ad-f30a-4343-8696-1ce5755a1c82-kube-api-access-sqc4f\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.330998 2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-sys\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.331124 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-sys\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.331243 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-podres\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.331287 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-proc\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.331385 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.331366 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f8084ad-f30a-4343-8696-1ce5755a1c82-lib-modules\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.342511 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.342477 2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqc4f\" (UniqueName: \"kubernetes.io/projected/2f8084ad-f30a-4343-8696-1ce5755a1c82-kube-api-access-sqc4f\") pod \"perf-node-gather-daemonset-rd67d\" (UID: \"2f8084ad-f30a-4343-8696-1ce5755a1c82\") " pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.361439 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.361401 2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.562166 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.554990 2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d"] Jul 22 12:35:37.565958 ip-10-0-143-104 kubenswrapper[2534]: W0722 12:35:37.565925 2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2f8084ad_f30a_4343_8696_1ce5755a1c82.slice/crio-5878db03e80a2da46b1c97afac8c680b710689100058c4e7dfc4a1e291c4507b WatchSource:0}: Error finding container 5878db03e80a2da46b1c97afac8c680b710689100058c4e7dfc4a1e291c4507b: Status 404 returned error can't find the container with id 5878db03e80a2da46b1c97afac8c680b710689100058c4e7dfc4a1e291c4507b Jul 22 12:35:37.813615 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.813579 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" event={"ID":"2f8084ad-f30a-4343-8696-1ce5755a1c82","Type":"ContainerStarted","Data":"5341ca5a68663e149de05975c31c6dbee50d7b4909144e12cddc7a7ea6bad391"} Jul 22 12:35:37.813835 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.813639 2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" event={"ID":"2f8084ad-f30a-4343-8696-1ce5755a1c82","Type":"ContainerStarted","Data":"5878db03e80a2da46b1c97afac8c680b710689100058c4e7dfc4a1e291c4507b"} Jul 22 12:35:37.813835 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.813813 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:37.834004 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:37.833897 2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" podStartSLOduration=0.833876456 podStartE2EDuration="833.876456ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:35:37.833103448 +0000 UTC m=+2766.158950713" watchObservedRunningTime="2026-07-22 12:35:37.833876456 +0000 UTC m=+2766.159723722" Jul 22 12:35:38.435652 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:38.435619 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-wxrfc_ed4af996-0621-404d-b718-a5f7336fa640/networking-console-plugin/0.log" Jul 22 12:35:39.038340 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:39.038309 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/1.log" Jul 22 12:35:39.043500 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:39.043475 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74m8f_2be84eee-fb05-4086-9c78-d70bf3a3f00e/console-operator/2.log" Jul 22 12:35:39.608842 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:39.608815 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-4d8kg_eedfa2bc-04ce-41e3-bfb7-0e6c3a5e3fe2/download-server/0.log" Jul 22 12:35:40.257211 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:40.257180 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-x6bsw_dea87c46-b392-4a6d-97cb-116c1d213d84/volume-data-source-validator/0.log" Jul 22 12:35:41.488223 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:41.488196 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k494z_50f09e4c-b2c9-40e7-a509-278f3b5fcb63/dns/0.log" Jul 22 12:35:41.516627 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:41.516601 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k494z_50f09e4c-b2c9-40e7-a509-278f3b5fcb63/kube-rbac-proxy/0.log" Jul 22 12:35:41.625397 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:41.625363 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngkzd_03f2645b-2fc8-4bef-9bec-128198246ad6/dns-node-resolver/0.log" Jul 22 12:35:42.297129 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:42.297102 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4c6jg_1fdb909c-8893-43f5-ba58-a8f774076194/node-ca/0.log" Jul 22 12:35:43.430530 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:43.430489 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47p247_54d19681-f9d1-42bd-b173-04f59eea7835/istio-proxy/0.log" Jul 22 12:35:43.664922 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:43.664882 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/0.log" Jul 22 12:35:43.698667 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:43.698595 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/1.log" Jul 22 12:35:43.821026 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:43.821001 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79ccd9f656-4kgpj_8b58a72c-7ded-4849-a1fb-df32c70b6081/router/0.log" Jul 22 12:35:43.832235 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:43.832210 2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mtjsz/perf-node-gather-daemonset-rd67d" Jul 22 12:35:44.610070 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:44.610039 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wm925_05e907e6-7648-4881-9d40-a4ffffd9d2c5/serve-healthcheck-canary/0.log" Jul 22 12:35:45.125596 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:45.125564 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-656k7_ecf76870-fbfb-4e3f-80ad-7c83e7540f88/insights-operator/0.log" Jul 22 12:35:45.126371 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:45.126343 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-656k7_ecf76870-fbfb-4e3f-80ad-7c83e7540f88/insights-operator/1.log" Jul 22 12:35:45.251998 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:45.251973 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-787pj_b3ea5425-46df-4c5f-993e-0e82b7417cb8/kube-rbac-proxy/0.log" Jul 22 12:35:45.283956 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:45.283926 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-787pj_b3ea5425-46df-4c5f-993e-0e82b7417cb8/exporter/0.log" Jul 22 12:35:45.321447 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:45.321420 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-787pj_b3ea5425-46df-4c5f-993e-0e82b7417cb8/extractor/0.log" Jul 22 12:35:47.432683 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:47.432656 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47p247_54d19681-f9d1-42bd-b173-04f59eea7835/istio-proxy/0.log" Jul 22 12:35:47.603928 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:47.603901 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/0.log" Jul 22 12:35:47.638044 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:47.637990 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/1.log" Jul 22 12:35:47.735288 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:47.735210 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79ccd9f656-4kgpj_8b58a72c-7ded-4849-a1fb-df32c70b6081/router/0.log" Jul 22 12:35:48.629777 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:48.629745 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c4d6d56dd-cvj4c_779ecd9c-4776-4fd1-becc-dbf7127314b4/manager/0.log" Jul 22 12:35:48.684621 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:48.684587 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c4d6d56dd-cvj4c_779ecd9c-4776-4fd1-becc-dbf7127314b4/copy-manifests/0.log" Jul 22 12:35:48.957448 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:48.957340 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47p247_54d19681-f9d1-42bd-b173-04f59eea7835/istio-proxy/0.log" Jul 22 12:35:49.143735 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:49.143707 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/0.log" Jul 22 12:35:49.179468 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:49.179436 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9nrnq_74dd6e20-18d2-49c7-ba3f-26ad91a1c510/istio-proxy/1.log" Jul 22 12:35:49.266724 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:49.266650 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79ccd9f656-4kgpj_8b58a72c-7ded-4849-a1fb-df32c70b6081/router/0.log" Jul 22 12:35:50.184924 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:50.184886 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-755b658494-fj45k_f58d8345-352f-4c5e-b400-56dbc0546501/manager/0.log" Jul 22 12:35:50.213958 ip-10-0-143-104 kubenswrapper[2534]: I0722 12:35:50.213926 2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wzsln_8efb5175-8aa4-42b1-b041-106d38e61751/openshift-lws-operator/0.log"