Jul 09 15:31:23.327741 ip-10-0-133-0 systemd[1]: Starting Kubernetes Kubelet... Jul 09 15:31:23.803298 ip-10-0-133-0 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 15:31:23.803298 ip-10-0-133-0 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 15:31:23.803298 ip-10-0-133-0 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 15:31:23.803298 ip-10-0-133-0 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 15:31:23.803298 ip-10-0-133-0 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 15:31:23.806373 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.806278 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 15:31:23.812145 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812128 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:31:23.812145 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812145 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812149 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812152 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812157 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812161 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812164 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812169 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812173 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812176 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812180 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812183 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812186 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812189 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812192 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812195 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812197 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812200 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812209 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812211 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812214 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:31:23.812214 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812217 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812222 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812226 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812230 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812232 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812235 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812238 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812241 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812244 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812247 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812250 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812253 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812256 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812258 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812268 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812271 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812274 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812278 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812281 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:31:23.812730 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812284 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812288 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812291 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812294 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812296 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812300 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812303 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812306 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812309 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812311 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812314 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812316 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812319 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812322 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812325 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812328 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812331 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812333 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812336 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812339 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:31:23.813237 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812342 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812344 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812347 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812349 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812352 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812354 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812357 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812360 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812362 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812365 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812368 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812370 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812373 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812376 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812378 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812381 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812384 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812387 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812391 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812393 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:31:23.813763 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812397 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812400 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812402 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812405 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812432 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812437 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812440 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812443 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812447 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812450 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812453 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812456 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812458 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812461 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812464 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812467 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812469 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812472 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.812475 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813325 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:31:23.814264 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813337 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813341 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813344 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813348 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813352 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813355 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813359 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813364 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813367 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813370 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813373 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813375 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813378 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813381 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813385 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813389 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813393 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813396 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813399 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:31:23.814805 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813402 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813405 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813408 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813411 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813414 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813416 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813419 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813422 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813424 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813427 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813429 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813433 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813437 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813441 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813444 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813447 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813449 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813452 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813455 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:31:23.815497 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813457 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813460 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813463 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813467 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813470 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813474 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813477 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813479 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813482 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813485 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813488 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813491 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813493 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813496 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813498 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813501 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813504 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813506 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813510 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813513 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:31:23.816243 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813532 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813535 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813538 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813541 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813544 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813547 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813550 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813554 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813556 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813559 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813562 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813565 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813567 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813571 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813573 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813577 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813579 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813582 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813585 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813588 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:31:23.816768 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813590 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813593 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813596 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813598 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813601 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813604 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813606 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813609 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813611 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813614 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813617 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813620 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813623 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813625 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813628 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813631 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813634 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813637 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813640 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.813642 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:31:23.817280 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814049 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814054 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814057 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814059 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814062 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814065 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814068 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814070 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814073 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814075 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814078 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814081 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814084 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814086 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814090 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814092 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814095 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814098 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814102 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814106 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:31:23.817818 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814109 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814113 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814116 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814119 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814122 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814125 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814127 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814130 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814133 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814136 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814138 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814141 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814144 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814147 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814150 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814153 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814156 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814158 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814161 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814163 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:31:23.818354 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814166 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814168 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814171 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814174 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814176 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814179 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814181 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814184 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814187 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814189 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814192 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814195 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814197 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814201 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814203 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814206 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814208 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814211 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814214 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814216 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:31:23.818872 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814219 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814222 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814225 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814228 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814230 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814234 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814237 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814239 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814241 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814245 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814248 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814251 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814254 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814257 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814261 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814265 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814268 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814271 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814274 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814277 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:31:23.819383 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814280 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814283 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814286 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814289 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814292 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814296 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814298 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814301 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814304 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814306 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814309 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814312 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814318 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814320 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814323 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814326 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814329 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814331 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.814334 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814409 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 15:31:23.819921 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814417 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814424 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814429 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814433 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814436 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814440 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814444 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814448 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814451 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814455 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814459 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814462 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814465 2577 flags.go:64] FLAG: --cgroup-root="" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814468 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814471 2577 flags.go:64] FLAG: --client-ca-file="" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814474 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814478 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814483 2577 flags.go:64] FLAG: --cluster-domain="" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814486 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814489 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814492 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814496 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814500 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 15:31:23.820427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814503 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814508 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814511 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814515 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814518 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814521 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814524 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814527 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814532 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814535 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814555 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814560 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814566 2577 flags.go:64] FLAG: --enable-server="true" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814569 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814575 2577 flags.go:64] FLAG: --event-burst="100" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814578 2577 flags.go:64] FLAG: --event-qps="50" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814581 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814585 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814588 2577 flags.go:64] FLAG: --eviction-hard="" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814592 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814595 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814598 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814601 2577 flags.go:64] FLAG: --eviction-soft="" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814604 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814607 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 15:31:23.821027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814610 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814613 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814616 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814619 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814622 2577 flags.go:64] FLAG: --feature-gates="" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814626 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814629 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814633 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814637 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814641 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814644 2577 flags.go:64] FLAG: --help="false" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814647 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-133-0.ec2.internal" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814650 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814653 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814657 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814660 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814665 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814668 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814671 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814674 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814677 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814680 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814684 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814687 2577 flags.go:64] FLAG: --kube-reserved="" Jul 09 15:31:23.821748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814690 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814693 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814696 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814699 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814702 2577 flags.go:64] FLAG: --lock-file="" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814705 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814708 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814710 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814716 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814719 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814722 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814725 2577 flags.go:64] FLAG: --logging-format="text" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814728 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814732 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814735 2577 flags.go:64] FLAG: --manifest-url="" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814738 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814746 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814749 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814754 2577 flags.go:64] FLAG: --max-pods="110" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814757 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814761 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814769 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814772 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814775 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814778 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 15:31:23.822357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814782 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814789 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814793 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814796 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814799 2577 flags.go:64] FLAG: --pod-cidr="" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814804 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814810 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814813 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814816 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814820 2577 flags.go:64] FLAG: --port="10250" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814823 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814826 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0923096c6a3690b95" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814830 2577 flags.go:64] FLAG: --qos-reserved="" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814833 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814836 2577 flags.go:64] FLAG: --register-node="true" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814839 2577 flags.go:64] FLAG: --register-with-taints="" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814843 2577 flags.go:64] FLAG: --registry-burst="10" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814847 2577 flags.go:64] FLAG: --registry-qps="5" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814849 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814852 2577 flags.go:64] FLAG: --reserved-memory="" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814856 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814860 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814863 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814866 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814872 2577 flags.go:64] FLAG: --runonce="false" Jul 09 15:31:23.823023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814875 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814878 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814881 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814884 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814887 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814891 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814894 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814897 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814900 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814904 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814907 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814909 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814914 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814927 2577 flags.go:64] FLAG: --system-cgroups="" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814932 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814938 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814941 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814944 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814948 2577 flags.go:64] FLAG: --tls-min-version="" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814951 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814953 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814956 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814959 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814963 2577 flags.go:64] FLAG: --v="2" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814967 2577 flags.go:64] FLAG: --version="false" Jul 09 15:31:23.824040 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814971 2577 flags.go:64] FLAG: --vmodule="" Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814975 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.814979 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815406 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815429 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815435 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815444 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815449 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815454 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815459 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815464 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815469 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815473 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815478 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815488 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815493 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815500 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815505 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815511 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815517 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:31:23.825177 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815521 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815526 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815530 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815547 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815553 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815558 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815562 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815571 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815576 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815581 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815585 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815590 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815594 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815601 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815608 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815612 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815617 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815622 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815630 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:31:23.826246 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815641 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815645 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815650 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815654 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815658 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815663 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815667 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815671 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815676 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815680 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815685 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815689 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815701 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815705 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815710 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815714 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815719 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815723 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815728 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815733 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:31:23.827182 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815737 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815741 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815746 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815750 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815755 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815764 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815769 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815773 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815778 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815782 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815787 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815791 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815796 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815800 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815804 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815809 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815816 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815827 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815833 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815838 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:31:23.827822 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815843 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815848 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815853 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815858 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815864 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815869 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815873 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815878 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815883 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815888 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815897 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815902 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815906 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815911 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815915 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815920 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815924 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815928 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815934 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815938 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:31:23.828516 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815943 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815947 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.815957 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.816790 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.825738 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.825761 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.825790 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.825799 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825891 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825899 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825905 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825910 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825916 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825920 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:31:23.829389 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825925 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825929 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825936 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825941 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825945 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825950 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825954 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825959 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825964 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825968 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825973 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825977 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825982 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825986 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825991 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.825995 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826000 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826004 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826008 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:31:23.830062 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826013 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826020 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826027 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826032 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826039 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826044 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826048 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826054 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826059 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826063 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826068 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826072 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826077 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826082 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826086 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826091 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826096 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826100 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826104 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:31:23.830585 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826109 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826113 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826118 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826122 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826126 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826130 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826135 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826141 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826147 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826151 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826156 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826160 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826165 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826169 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826173 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826178 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826182 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826187 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826192 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826196 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:31:23.831192 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826201 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826206 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826211 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826216 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826220 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826225 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826232 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826238 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826243 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826248 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826252 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826256 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826260 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826264 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826270 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826274 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826280 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826284 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826289 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:31:23.831750 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826293 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826298 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826302 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826306 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826310 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826314 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826318 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826323 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826327 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826332 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826338 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826345 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826351 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826357 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826361 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:31:23.832267 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826366 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.826374 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826593 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826602 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826607 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826612 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826616 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826621 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826626 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826630 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826634 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826639 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826644 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826650 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826654 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 15:31:23.832720 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826659 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826663 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826668 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826673 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826677 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826682 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826687 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826691 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826695 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826700 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826705 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826709 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826714 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826720 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826724 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826729 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826734 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826738 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826743 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826747 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 15:31:23.833172 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826751 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826756 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826763 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826769 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826774 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826778 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826783 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826787 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826792 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826796 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826802 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826807 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826811 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826816 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826821 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826825 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826829 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826834 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826838 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 15:31:23.833726 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826842 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826847 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826851 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826856 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826860 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826865 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826870 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826874 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826879 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826883 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826887 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826892 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826896 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826901 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826905 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826910 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826915 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826921 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826925 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 15:31:23.834221 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826929 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826934 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826938 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826943 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826948 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826953 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826958 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826962 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826967 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826971 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826976 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826980 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826985 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826989 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826993 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.826998 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827003 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827007 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827011 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827016 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 15:31:23.834775 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827021 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827025 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827030 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827034 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827039 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827043 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827049 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:23.827055 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.827065 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.827267 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.831001 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.833019 2577 server.go:1019] "Starting client certificate rotation" Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.833136 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 15:31:23.835269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.833178 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 15:31:23.837980 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.837960 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 15:31:23.841720 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.841698 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 15:31:23.857283 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.857263 2577 log.go:25] "Validated CRI v1 runtime API" Jul 09 15:31:23.860172 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.860158 2577 log.go:25] "Validated CRI v1 image API" Jul 09 15:31:23.862106 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.862079 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 15:31:23.868857 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.868830 2577 fs.go:135] Filesystem UUIDs: map[2638ebf2-dff9-4ff6-afd0-2b5554cbc347:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 95e571d5-541e-4f35-91e4-e434fed457c1:/dev/nvme0n1p3] Jul 09 15:31:23.868934 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.868854 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 15:31:23.871446 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.871429 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 15:31:23.876145 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.876023 2577 manager.go:216] Machine: {Timestamp:2026-07-09 15:31:23.874627129 +0000 UTC m=+0.411854303 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3186878 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec258d950e92d30e2358c795f0962050 SystemUUID:ec258d95-0e92-d30e-2358-c795f0962050 BootID:315f12ab-1aa1-43c3-98b1-47a699570ccb Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:65:a7:b8:cb:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:65:a7:b8:cb:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:fb:53:6c:ba:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 15:31:23.876145 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.876142 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 15:31:23.876287 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.876275 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 15:31:23.877776 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.877751 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 15:31:23.877923 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.877780 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-0.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 15:31:23.879659 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.879648 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 15:31:23.879694 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.879663 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 15:31:23.879694 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.879671 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 15:31:23.880523 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.880512 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 15:31:23.882837 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.882821 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 15:31:23.888170 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888158 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 09 15:31:23.888300 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888290 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 15:31:23.888330 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888323 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 09 15:31:23.888358 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888333 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 15:31:23.888358 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888349 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 15:31:23.888358 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888358 2577 kubelet.go:398] "Adding apiserver pod source" Jul 09 15:31:23.888458 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888367 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 15:31:23.888775 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888761 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 15:31:23.888844 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.888782 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 15:31:23.902584 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.902564 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 15:31:23.905356 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905341 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 15:31:23.905431 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905374 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 15:31:23.905600 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905588 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 15:31:23.905637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905607 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 15:31:23.905637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905613 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 15:31:23.905637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905619 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 15:31:23.905637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905628 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 15:31:23.905637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.905634 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 15:31:23.906238 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.906227 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 15:31:23.906275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.906238 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 15:31:23.906275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.906247 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 15:31:23.906275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.906264 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 15:31:23.906358 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.906279 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 15:31:23.906358 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.906287 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 15:31:23.908060 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.908048 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 15:31:23.908105 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.908061 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 15:31:23.910012 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.909990 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 15:31:23.910063 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.910001 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 15:31:23.911647 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.911634 2577 server.go:1268] "Started kubelet" Jul 09 15:31:23.911804 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.911732 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 15:31:23.912051 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.911743 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 15:31:23.912148 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.912075 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 15:31:23.912758 ip-10-0-133-0 systemd[1]: Started Kubernetes Kubelet. Jul 09 15:31:23.913914 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.913860 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 15:31:23.914568 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.914553 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 09 15:31:23.920005 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.919978 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 15:31:23.921037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.921019 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-0.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 15:31:23.922053 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.921004 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b2002210 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.911602704 +0000 UTC m=+0.448829872,LastTimestamp:2026-07-09 15:31:23.911602704 +0000 UTC m=+0.448829872,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:23.924299 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.924280 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 15:31:23.924399 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.924299 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 15:31:23.925264 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.924944 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:23.925482 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925339 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 15:31:23.925482 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925454 2577 factory.go:55] Registering systemd factory Jul 09 15:31:23.925482 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925471 2577 factory.go:223] Registration of the systemd container factory successfully Jul 09 15:31:23.925670 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925488 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 15:31:23.925670 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925520 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 15:31:23.925670 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925594 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 09 15:31:23.925670 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925603 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 09 15:31:23.925947 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925931 2577 factory.go:153] Registering CRI-O factory Jul 09 15:31:23.926001 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.925950 2577 factory.go:223] Registration of the crio container factory successfully Jul 09 15:31:23.926055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.926007 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 15:31:23.926055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.926034 2577 factory.go:103] Registering Raw factory Jul 09 15:31:23.926055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.926050 2577 manager.go:1196] Started watching for new ooms in manager Jul 09 15:31:23.927113 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.927091 2577 manager.go:349] Starting recovery of all containers Jul 09 15:31:23.927714 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.927688 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 15:31:23.927823 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.927795 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 15:31:23.940715 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.940694 2577 manager.go:354] Recovery completed Jul 09 15:31:23.945072 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.945059 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:23.947485 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.947455 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:23.947587 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.947501 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:23.947587 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.947526 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:23.948113 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948099 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 15:31:23.948113 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948112 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 15:31:23.948196 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948129 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 09 15:31:23.948196 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948151 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:23.948921 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948900 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:23.949007 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948941 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:23.949007 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.948952 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:23.949735 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.949719 2577 policy_none.go:49] "None policy: Start" Jul 09 15:31:23.949735 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.949738 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 15:31:23.949880 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.949749 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 15:31:23.951167 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.951152 2577 policy_none.go:47] "Start" Jul 09 15:31:23.953743 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.953674 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:23.967132 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.967051 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:23.980456 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.980390 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:23.993479 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.993396 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:23.948925485 +0000 UTC m=+0.486152652,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:23.997195 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.997181 2577 manager.go:337] "Starting Device Plugin manager" Jul 09 15:31:23.997247 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.997215 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 15:31:23.997247 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.997225 2577 server.go:87] "Starting device plugin registration server" Jul 09 15:31:23.997533 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.997520 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 15:31:23.998202 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.998167 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 15:31:23.998418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.998294 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 15:31:23.998418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.998401 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 15:31:23.998418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:23.998412 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 15:31:23.998965 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.998938 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 15:31:23.999023 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:23.998995 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:24.007133 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.007059 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:23.948946927 +0000 UTC m=+0.486174093,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.018077 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.017986 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:23.94895636 +0000 UTC m=+0.486183528,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.058491 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.029112 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b73d6f61 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.999506273 +0000 UTC m=+0.536733427,LastTimestamp:2026-07-09 15:31:23.999506273 +0000 UTC m=+0.536733427,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.102931 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.102896 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.103944 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.103924 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.104021 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.103961 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.104021 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.103981 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.104021 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.104011 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.115206 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.115128 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:24.103944334 +0000 UTC m=+0.641171500,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.128117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.128074 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 15:31:24.129428 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.129407 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 15:31:24.129519 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.129452 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 15:31:24.129519 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.129474 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 15:31:24.129616 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.129573 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 15:31:24.130386 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.130287 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:24.10396847 +0000 UTC m=+0.641195638,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.130497 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.130400 2577 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.130579 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.130484 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 09 15:31:24.133429 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.133361 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:24.103985839 +0000 UTC m=+0.641213006,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.133577 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.133558 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 09 15:31:24.230740 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.230690 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal"] Jul 09 15:31:24.232790 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.230785 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.233291 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.233269 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.233349 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.233311 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.233349 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.233323 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.235052 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235039 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.235195 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235180 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.235241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235216 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.235885 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235863 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.235985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235896 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.235985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235912 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.235985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235929 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.235985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235960 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.235985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.235975 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.236220 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.236008 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:24.233295289 +0000 UTC m=+0.770522461,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.237424 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.237402 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.237535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.237434 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.238139 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.238124 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.238231 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.238153 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.238231 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.238167 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.253439 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.253374 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:24.23331772 +0000 UTC m=+0.770544886,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.263990 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.263924 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:24.233327541 +0000 UTC m=+0.770554708,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.273323 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.273305 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-0.ec2.internal\" not found" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.275119 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.275054 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:24.235881272 +0000 UTC m=+0.773108438,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.278117 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.278098 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-0.ec2.internal\" not found" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.288675 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.288602 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:24.235904637 +0000 UTC m=+0.773131805,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.301477 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.301404 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:24.235918776 +0000 UTC m=+0.773145948,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.309855 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.309769 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:24.235947604 +0000 UTC m=+0.773174774,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.321320 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.321260 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:24.235966831 +0000 UTC m=+0.773193999,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.327844 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.327824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0df129d73aab3b87bad4f6dc5940ca16-config\") pod \"kube-apiserver-proxy-ip-10-0-133-0.ec2.internal\" (UID: \"0df129d73aab3b87bad4f6dc5940ca16\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.327901 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.327850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/07c2d5eaa8bf47df92ffe81e309b9062-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" (UID: \"07c2d5eaa8bf47df92ffe81e309b9062\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.327901 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.327869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07c2d5eaa8bf47df92ffe81e309b9062-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" (UID: \"07c2d5eaa8bf47df92ffe81e309b9062\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.331189 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.331176 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.332112 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.332086 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.332228 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.332125 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.332228 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.332139 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.332228 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.332173 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.336143 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.336085 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:24.235980822 +0000 UTC m=+0.773207990,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.345113 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.345086 2577 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.345220 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.345098 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:24.238137629 +0000 UTC m=+0.775364807,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.360900 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.360823 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:24.238160506 +0000 UTC m=+0.775387674,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.372633 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.372566 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b42458ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947530495 +0000 UTC m=+0.484757661,LastTimestamp:2026-07-09 15:31:24.238172716 +0000 UTC m=+0.775399884,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.385479 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.385396 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b423a0b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947483318 +0000 UTC m=+0.484710487,LastTimestamp:2026-07-09 15:31:24.332107677 +0000 UTC m=+0.869334849,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.396753 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.396690 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-0.ec2.internal.18c0a8d4b4242ca2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-0.ec2.internal,UID:ip-10-0-133-0.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-0.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:23.947519138 +0000 UTC m=+0.484746305,LastTimestamp:2026-07-09 15:31:24.332131756 +0000 UTC m=+0.869358927,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:24.428392 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.428363 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/07c2d5eaa8bf47df92ffe81e309b9062-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" (UID: \"07c2d5eaa8bf47df92ffe81e309b9062\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.428501 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.428400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07c2d5eaa8bf47df92ffe81e309b9062-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" (UID: \"07c2d5eaa8bf47df92ffe81e309b9062\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.428501 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.428417 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0df129d73aab3b87bad4f6dc5940ca16-config\") pod \"kube-apiserver-proxy-ip-10-0-133-0.ec2.internal\" (UID: \"0df129d73aab3b87bad4f6dc5940ca16\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.428501 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.428469 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07c2d5eaa8bf47df92ffe81e309b9062-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" (UID: \"07c2d5eaa8bf47df92ffe81e309b9062\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.428501 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.428492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/07c2d5eaa8bf47df92ffe81e309b9062-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" (UID: \"07c2d5eaa8bf47df92ffe81e309b9062\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.428667 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.428560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0df129d73aab3b87bad4f6dc5940ca16-config\") pod \"kube-apiserver-proxy-ip-10-0-133-0.ec2.internal\" (UID: \"0df129d73aab3b87bad4f6dc5940ca16\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.541163 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.541125 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms" Jul 09 15:31:24.578185 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.578118 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.581467 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.581449 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.745856 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.745819 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:24.746939 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.746917 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:24.747035 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.746954 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:24.747035 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.746965 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:24.747035 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.746993 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.772441 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:24.772407 2577 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:24.934113 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:24.934082 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-0.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 15:31:25.071142 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.071104 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 09 15:31:25.121319 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.121288 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:25.122327 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.122303 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:25.122445 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.122339 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:25.122445 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.122350 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:25.133848 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.133801 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 15:31:25.133848 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.133822 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 15:31:25.286150 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.285935 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 15:31:25.289296 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:25.289257 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df129d73aab3b87bad4f6dc5940ca16.slice/crio-2aef1bed2048f24c223315f4033f56e7f45bc1e5e7196eae0a017540ed41f635 WatchSource:0}: Error finding container 2aef1bed2048f24c223315f4033f56e7f45bc1e5e7196eae0a017540ed41f635: Status 404 returned error can't find the container with id 2aef1bed2048f24c223315f4033f56e7f45bc1e5e7196eae0a017540ed41f635 Jul 09 15:31:25.289821 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:25.289791 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c2d5eaa8bf47df92ffe81e309b9062.slice/crio-70f9504b63fdfd1981817d67d3217827e42af03876c289f00d1c43574b7695b6 WatchSource:0}: Error finding container 70f9504b63fdfd1981817d67d3217827e42af03876c289f00d1c43574b7695b6: Status 404 returned error can't find the container with id 70f9504b63fdfd1981817d67d3217827e42af03876c289f00d1c43574b7695b6 Jul 09 15:31:25.295104 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.295089 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:31:25.306329 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.306248 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal.18c0a8d504793fea openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal,UID:07c2d5eaa8bf47df92ffe81e309b9062,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d1573db16143027c8ffaf46a01ce12f2bcd89c15410c42ca54be4b3b1ddfdd6\",Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:25.295271914 +0000 UTC m=+1.832499081,LastTimestamp:2026-07-09 15:31:25.295271914 +0000 UTC m=+1.832499081,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:25.316476 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.316381 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-133-0.ec2.internal.18c0a8d5047a29dc kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-133-0.ec2.internal,UID:0df129d73aab3b87bad4f6dc5940ca16,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0595b1d6de8e08ddb61ce581075357331193db6206d341c15cf213514ef0f693\",Source:EventSource{Component:kubelet,Host:ip-10-0-133-0.ec2.internal,},FirstTimestamp:2026-07-09 15:31:25.295331804 +0000 UTC m=+1.832558957,LastTimestamp:2026-07-09 15:31:25.295331804 +0000 UTC m=+1.832558957,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-0.ec2.internal,}" Jul 09 15:31:25.352635 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.352602 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s" Jul 09 15:31:25.573200 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.573084 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:25.574311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.574291 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:25.574422 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.574326 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:25.574422 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.574336 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:25.574422 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.574370 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:25.598945 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:25.598915 2577 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:25.859411 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.859331 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hv8g7" Jul 09 15:31:25.868897 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.868873 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hv8g7" Jul 09 15:31:25.939900 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.939865 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:25.959496 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:25.959428 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.019872 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.019840 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.138603 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.138531 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" event={"ID":"0df129d73aab3b87bad4f6dc5940ca16","Type":"ContainerStarted","Data":"2aef1bed2048f24c223315f4033f56e7f45bc1e5e7196eae0a017540ed41f635"} Jul 09 15:31:26.140398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.140360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" event={"ID":"07c2d5eaa8bf47df92ffe81e309b9062","Type":"ContainerStarted","Data":"70f9504b63fdfd1981817d67d3217827e42af03876c289f00d1c43574b7695b6"} Jul 09 15:31:26.290712 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.290681 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.290712 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:26.290715 2577 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.331100 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.331063 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.350726 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.350696 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.414067 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.413965 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.695829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.695747 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.695829 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:26.695782 2577 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.832807 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.832768 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 15:31:26.874367 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.874308 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 15:26:26 +0000 UTC" deadline="2028-01-13 05:37:00.149485257 +0000 UTC" Jul 09 15:31:26.874367 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.874356 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13262h5m33.275133762s" Jul 09 15:31:26.962767 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.962678 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:26.962767 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:26.962700 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-0.ec2.internal\" not found" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:26.979868 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:26.979836 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:27.037798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.037766 2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-0.ec2.internal" not found Jul 09 15:31:27.111813 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.111781 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:31:27.172503 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.172458 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:27.173565 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.173527 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:27.173728 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.173582 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:27.173728 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.173597 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:27.199760 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.199723 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 15:31:27.200721 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.200700 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientMemory" Jul 09 15:31:27.200846 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.200737 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 15:31:27.200846 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.200751 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeHasSufficientPID" Jul 09 15:31:27.200846 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.200788 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:27.212155 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.212134 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-0.ec2.internal" Jul 09 15:31:27.212263 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.212179 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-0.ec2.internal\": node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.226934 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.226870 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.327966 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.327932 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.429115 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.429075 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.529165 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.529145 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.629935 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.629884 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.730420 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.730393 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.773284 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:27.773259 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:31:27.831112 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.831077 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:27.932101 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:27.932069 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:28.032701 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:28.032637 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-0.ec2.internal\" not found" Jul 09 15:31:28.083246 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.083221 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:31:28.124919 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.124882 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:28.135761 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.135740 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:31:28.136614 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.136602 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:28.143813 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.143798 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:31:28.144731 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.144707 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" event={"ID":"0df129d73aab3b87bad4f6dc5940ca16","Type":"ContainerStarted","Data":"c263bc477143f2333ba808a17e37fbc54527a9b0ffdfaa9b0fbdb91b315dec20"} Jul 09 15:31:28.144965 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.144950 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:28.146376 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.146354 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" event={"ID":"07c2d5eaa8bf47df92ffe81e309b9062","Type":"ContainerStarted","Data":"b1655a385e874135d51cb112126db9633abaf0d62d7d964dfc1e04e112247847"} Jul 09 15:31:28.146544 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.146532 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:28.156806 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.156790 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:31:28.156868 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:28.156827 2577 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-0.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:28.157411 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.157398 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:31:28.157454 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:28.157430 2577 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:28.218290 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.218266 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 15:31:28.892759 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.892729 2577 apiserver.go:52] "Watching apiserver" Jul 09 15:31:28.898569 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.898545 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 15:31:28.900975 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.900952 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-np5vh","openshift-network-diagnostics/network-check-target-wx6lp","openshift-network-operator/iptables-alerter-9hvvb","openshift-cluster-node-tuning-operator/tuned-4pfnr","openshift-dns/node-resolver-4zmmk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal","openshift-multus/multus-glxth","openshift-multus/network-metrics-daemon-s7k7t","openshift-ovn-kubernetes/ovnkube-node-pj2r4","kube-system/global-pull-secret-syncer-8nz45","kube-system/konnectivity-agent-9m4mg","kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll","openshift-image-registry/node-ca-cz222"] Jul 09 15:31:28.902576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.902560 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-glxth" Jul 09 15:31:28.904112 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.904094 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.904588 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.904571 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.904728 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.904711 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w8mtk\"" Jul 09 15:31:28.904803 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.904787 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 15:31:28.904847 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.904803 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.904884 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.904867 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 15:31:28.905282 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.905269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:28.905776 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.905758 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.905844 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.905814 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-njzql\"" Jul 09 15:31:28.905844 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.905834 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.905975 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.905960 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 15:31:28.906767 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.906744 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 15:31:28.906767 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.906764 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.907022 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.907004 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.907095 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.907080 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.907314 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.907295 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l8cv6\"" Jul 09 15:31:28.907940 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.907922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:28.909374 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.909349 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.909477 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.909447 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.911107 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.909937 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.911107 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.909965 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tbb8h\"" Jul 09 15:31:28.911107 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.910083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:28.911107 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.910183 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.911107 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.911024 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xfpjk\"" Jul 09 15:31:28.911981 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.911956 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wl7hn\"" Jul 09 15:31:28.912080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.911958 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.912142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.912097 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 15:31:28.912422 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.912402 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.912723 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.912431 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.914174 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.914153 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.914258 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.914246 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 15:31:28.914499 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.914438 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 15:31:28.914584 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.914557 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 15:31:28.914632 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.914622 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 15:31:28.914712 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.914696 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 15:31:28.915080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.915047 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pk4qz\"" Jul 09 15:31:28.915188 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.915171 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 15:31:28.915715 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.915698 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 15:31:28.915829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.915802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:28.915896 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:28.915871 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:28.916095 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.916081 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qmmn6\"" Jul 09 15:31:28.916179 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.916150 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 15:31:28.917330 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.917305 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:28.917402 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:28.917369 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:28.919198 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.919186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:28.920841 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.920824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:28.920923 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:28.920874 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:28.920923 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.920897 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 15:31:28.921025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.920987 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 15:31:28.921068 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.921038 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5jh8v\"" Jul 09 15:31:28.926314 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.926298 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 15:31:28.927717 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.927698 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 15:31:28.937698 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.937664 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 15:31:28.950660 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.950613 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" podStartSLOduration=0.950592632 podStartE2EDuration="950.592632ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:28.935406908 +0000 UTC m=+5.472634077" watchObservedRunningTime="2026-07-09 15:31:28.950592632 +0000 UTC m=+5.487819805" Jul 09 15:31:28.953818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.953800 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/27e8fecd-4af3-48b4-af25-7636492aac49-multus-daemon-config\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.953903 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.953831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-multus-certs\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.953903 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.953855 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-etc-kubernetes\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.953972 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.953913 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-dbus\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:28.953972 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.953946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-hostroot\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.953980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-socket-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.954033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ffs\" (UniqueName: \"kubernetes.io/projected/9d88c134-1d09-4f69-86a6-6d13c89a08b2-kube-api-access-v4ffs\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.954033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954022 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-ovn\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954039 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-system-cni-dir\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.954141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-cnibin\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.954141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-iptables-alerter-script\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:28.954141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954117 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysctl-d\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-run\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-conf-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-node-log\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovnkube-config\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovn-node-metrics-cert\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954286 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-sys\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.954311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-cni-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954334 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-cni-multus\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27e8fecd-4af3-48b4-af25-7636492aac49-cni-binary-copy\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-k8s-cni-cncf-io\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-kubelet\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954445 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-run-netns\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-cni-binary-copy\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954515 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-registration-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954538 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-systemd\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954559 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-os-release\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-kubernetes\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954612 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-var-lib-kubelet\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-kubelet\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.954689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-etc-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954700 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysconfig\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f176317-d27b-4f21-b235-359b94766bbd-konnectivity-ca\") pod \"konnectivity-agent-9m4mg\" (UID: \"5f176317-d27b-4f21-b235-359b94766bbd\") " pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44fnr\" (UniqueName: \"kubernetes.io/projected/27e8fecd-4af3-48b4-af25-7636492aac49-kube-api-access-44fnr\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954799 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovnkube-script-lib\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9hfr\" (UniqueName: \"kubernetes.io/projected/1c3ed855-db93-4389-b437-eb12d222712b-kube-api-access-g9hfr\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-cnibin\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954910 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-sys-fs\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954933 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/45f9fd98-9365-48d7-a594-1ca287fd2234-hosts-file\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74z8j\" (UniqueName: \"kubernetes.io/projected/45f9fd98-9365-48d7-a594-1ca287fd2234-kube-api-access-74z8j\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.954980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-host\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45f9fd98-9365-48d7-a594-1ca287fd2234-tmp-dir\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955064 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-cni-netd\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysctl-conf\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955129 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-tuned\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e873854a-76a3-4172-ae5e-bbd3af1a04b7-host\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:28.955254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955170 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-var-lib-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-log-socket\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr95b\" (UniqueName: \"kubernetes.io/projected/32759466-2823-4980-a821-990d82f7b87b-kube-api-access-qr95b\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-systemd-units\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-kubelet-config\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-host-slash\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955328 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd24d\" (UniqueName: \"kubernetes.io/projected/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-kube-api-access-vd24d\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-system-cni-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-slash\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-modprobe-d\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-device-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e873854a-76a3-4172-ae5e-bbd3af1a04b7-serviceca\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:28.955967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955603 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-lib-modules\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955655 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-cni-bin\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-cni-bin\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-env-overrides\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shvpb\" (UniqueName: \"kubernetes.io/projected/547d60d5-feb0-45c1-af93-9eb6d5b593c4-kube-api-access-shvpb\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-systemd\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-tmp\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f176317-d27b-4f21-b235-359b94766bbd-agent-certs\") pod \"konnectivity-agent-9m4mg\" (UID: \"5f176317-d27b-4f21-b235-359b94766bbd\") " pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-os-release\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955889 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-socket-dir-parent\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-netns\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:28.956438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:28.956874 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qszd6\" (UniqueName: \"kubernetes.io/projected/e873854a-76a3-4172-ae5e-bbd3af1a04b7-kube-api-access-qszd6\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:28.956874 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.955973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qxn\" (UniqueName: \"kubernetes.io/projected/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-kube-api-access-j6qxn\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:28.959120 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.959103 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fjcxx" Jul 09 15:31:28.966550 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:28.966534 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fjcxx" Jul 09 15:31:29.056535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-registration-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.056535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056529 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-systemd\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.056535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-os-release\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-kubernetes\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056578 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-var-lib-kubelet\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-kubelet\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056621 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-etc-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056618 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-systemd\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056652 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-os-release\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-var-lib-kubelet\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056678 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-kubelet\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056679 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-etc-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-kubernetes\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysconfig\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f176317-d27b-4f21-b235-359b94766bbd-konnectivity-ca\") pod \"konnectivity-agent-9m4mg\" (UID: \"5f176317-d27b-4f21-b235-359b94766bbd\") " pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysconfig\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056736 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-registration-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44fnr\" (UniqueName: \"kubernetes.io/projected/27e8fecd-4af3-48b4-af25-7636492aac49-kube-api-access-44fnr\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.057116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056809 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovnkube-script-lib\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9hfr\" (UniqueName: \"kubernetes.io/projected/1c3ed855-db93-4389-b437-eb12d222712b-kube-api-access-g9hfr\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-cnibin\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-sys-fs\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/45f9fd98-9365-48d7-a594-1ca287fd2234-hosts-file\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056896 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-cnibin\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056896 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74z8j\" (UniqueName: \"kubernetes.io/projected/45f9fd98-9365-48d7-a594-1ca287fd2234-kube-api-access-74z8j\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056958 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/45f9fd98-9365-48d7-a594-1ca287fd2234-hosts-file\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.056990 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-host\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45f9fd98-9365-48d7-a594-1ca287fd2234-tmp-dir\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-cni-netd\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057050 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-sys-fs\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057050 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-host\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057093 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-cni-netd\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysctl-conf\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-tuned\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e873854a-76a3-4172-ae5e-bbd3af1a04b7-host\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.057938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-var-lib-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e873854a-76a3-4172-ae5e-bbd3af1a04b7-host\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057210 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-log-socket\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057237 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-var-lib-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-log-socket\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057289 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr95b\" (UniqueName: \"kubernetes.io/projected/32759466-2823-4980-a821-990d82f7b87b-kube-api-access-qr95b\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057295 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysctl-conf\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-systemd-units\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057342 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f176317-d27b-4f21-b235-359b94766bbd-konnectivity-ca\") pod \"konnectivity-agent-9m4mg\" (UID: \"5f176317-d27b-4f21-b235-359b94766bbd\") " pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-kubelet-config\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057455 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-host-slash\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd24d\" (UniqueName: \"kubernetes.io/projected/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-kube-api-access-vd24d\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-system-cni-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057552 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovnkube-script-lib\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.058743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057570 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-slash\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-modprobe-d\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-systemd-units\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057602 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-host-slash\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-device-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-system-cni-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e873854a-76a3-4172-ae5e-bbd3af1a04b7-serviceca\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.057710 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057759 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.057806 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:31:29.557774892 +0000 UTC m=+6.095002057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057807 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-kubelet-config\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057802 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-slash\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-openvswitch\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-modprobe-d\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.060085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057949 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-device-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.057727 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.058008 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.058046 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:31:29.558032594 +0000 UTC m=+6.095259745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058061 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-lib-modules\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058063 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-cni-bin\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058108 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-cni-bin\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-env-overrides\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058170 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-cni-bin\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058210 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-lib-modules\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-cni-bin\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shvpb\" (UniqueName: \"kubernetes.io/projected/547d60d5-feb0-45c1-af93-9eb6d5b593c4-kube-api-access-shvpb\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.060909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-systemd\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-tmp\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058410 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f176317-d27b-4f21-b235-359b94766bbd-agent-certs\") pod \"konnectivity-agent-9m4mg\" (UID: \"5f176317-d27b-4f21-b235-359b94766bbd\") " pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-os-release\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-env-overrides\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058487 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-socket-dir-parent\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058496 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-systemd\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-netns\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058528 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qszd6\" (UniqueName: \"kubernetes.io/projected/e873854a-76a3-4172-ae5e-bbd3af1a04b7-kube-api-access-qszd6\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058597 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-socket-dir-parent\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qxn\" (UniqueName: \"kubernetes.io/projected/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-kube-api-access-j6qxn\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/27e8fecd-4af3-48b4-af25-7636492aac49-multus-daemon-config\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058653 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-os-release\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-multus-certs\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.061627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058739 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-multus-certs\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058747 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45f9fd98-9365-48d7-a594-1ca287fd2234-tmp-dir\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-etc-kubernetes\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058836 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-dbus\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-hostroot\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-socket-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058934 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-dbus\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ffs\" (UniqueName: \"kubernetes.io/projected/9d88c134-1d09-4f69-86a6-6d13c89a08b2-kube-api-access-v4ffs\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058968 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-ovn\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058984 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e873854a-76a3-4172-ae5e-bbd3af1a04b7-serviceca\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-system-cni-dir\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058995 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-etc-kubernetes\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.058892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-netns\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059023 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-run-ovn\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-cnibin\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-iptables-alerter-script\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059060 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-hostroot\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059060 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-system-cni-dir\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059108 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysctl-d\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-run\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059138 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32759466-2823-4980-a821-990d82f7b87b-cnibin\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d88c134-1d09-4f69-86a6-6d13c89a08b2-socket-dir\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-run\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059196 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-conf-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-node-log\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059229 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-sysctl-d\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-conf-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovnkube-config\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovn-node-metrics-cert\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-node-log\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-sys\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059342 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-sys\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059344 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-cni-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-cni-multus\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-multus-cni-dir\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.062661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27e8fecd-4af3-48b4-af25-7636492aac49-cni-binary-copy\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-k8s-cni-cncf-io\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-cni-multus\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059454 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-kubelet\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-run-k8s-cni-cncf-io\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-run-netns\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-cni-binary-copy\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059523 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27e8fecd-4af3-48b4-af25-7636492aac49-host-var-lib-kubelet\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059599 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/547d60d5-feb0-45c1-af93-9eb6d5b593c4-host-run-netns\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059953 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27e8fecd-4af3-48b4-af25-7636492aac49-cni-binary-copy\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.059964 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/27e8fecd-4af3-48b4-af25-7636492aac49-multus-daemon-config\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.060087 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32759466-2823-4980-a821-990d82f7b87b-cni-binary-copy\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.060192 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-iptables-alerter-script\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.060306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovnkube-config\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.061603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-tmp\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.061789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f176317-d27b-4f21-b235-359b94766bbd-agent-certs\") pod \"konnectivity-agent-9m4mg\" (UID: \"5f176317-d27b-4f21-b235-359b94766bbd\") " pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.061884 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-etc-tuned\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.063133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.062724 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/547d60d5-feb0-45c1-af93-9eb6d5b593c4-ovn-node-metrics-cert\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.067620 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.067024 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:31:29.067620 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.067061 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:31:29.067620 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.067093 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:29.067620 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.067247 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:31:29.567197969 +0000 UTC m=+6.104425122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:29.068144 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.068018 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9hfr\" (UniqueName: \"kubernetes.io/projected/1c3ed855-db93-4389-b437-eb12d222712b-kube-api-access-g9hfr\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:29.068241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.068218 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr95b\" (UniqueName: \"kubernetes.io/projected/32759466-2823-4980-a821-990d82f7b87b-kube-api-access-qr95b\") pod \"multus-additional-cni-plugins-np5vh\" (UID: \"32759466-2823-4980-a821-990d82f7b87b\") " pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.068408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.068386 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd24d\" (UniqueName: \"kubernetes.io/projected/1aa1ce07-ade0-407b-a5fa-9785a622f1b9-kube-api-access-vd24d\") pod \"iptables-alerter-9hvvb\" (UID: \"1aa1ce07-ade0-407b-a5fa-9785a622f1b9\") " pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.068510 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.068484 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74z8j\" (UniqueName: \"kubernetes.io/projected/45f9fd98-9365-48d7-a594-1ca287fd2234-kube-api-access-74z8j\") pod \"node-resolver-4zmmk\" (UID: \"45f9fd98-9365-48d7-a594-1ca287fd2234\") " pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.069018 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.068997 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shvpb\" (UniqueName: \"kubernetes.io/projected/547d60d5-feb0-45c1-af93-9eb6d5b593c4-kube-api-access-shvpb\") pod \"ovnkube-node-pj2r4\" (UID: \"547d60d5-feb0-45c1-af93-9eb6d5b593c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.069950 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.069933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44fnr\" (UniqueName: \"kubernetes.io/projected/27e8fecd-4af3-48b4-af25-7636492aac49-kube-api-access-44fnr\") pod \"multus-glxth\" (UID: \"27e8fecd-4af3-48b4-af25-7636492aac49\") " pod="openshift-multus/multus-glxth" Jul 09 15:31:29.070143 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.070124 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qxn\" (UniqueName: \"kubernetes.io/projected/caea8bbb-2a0c-49be-b64a-dbfd3fe9001c-kube-api-access-j6qxn\") pod \"tuned-4pfnr\" (UID: \"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c\") " pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.070275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.070257 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ffs\" (UniqueName: \"kubernetes.io/projected/9d88c134-1d09-4f69-86a6-6d13c89a08b2-kube-api-access-v4ffs\") pod \"aws-ebs-csi-driver-node-k8kll\" (UID: \"9d88c134-1d09-4f69-86a6-6d13c89a08b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.070307 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.070273 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qszd6\" (UniqueName: \"kubernetes.io/projected/e873854a-76a3-4172-ae5e-bbd3af1a04b7-kube-api-access-qszd6\") pod \"node-ca-cz222\" (UID: \"e873854a-76a3-4172-ae5e-bbd3af1a04b7\") " pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.147627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.147552 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:29.147759 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.147644 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:29.158530 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.158511 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:31:29.158650 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.158554 2577 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-0.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-0.ec2.internal" Jul 09 15:31:29.159274 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.159262 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 15:31:29.159320 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.159291 2577 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" Jul 09 15:31:29.214595 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.214569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-glxth" Jul 09 15:31:29.220080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.220063 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" Jul 09 15:31:29.226020 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.225995 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e8fecd_4af3_48b4_af25_7636492aac49.slice/crio-55e3070ba193b5bf6a529fcad06bc5c997eb44d91d258edf28a977b39a99077a WatchSource:0}: Error finding container 55e3070ba193b5bf6a529fcad06bc5c997eb44d91d258edf28a977b39a99077a: Status 404 returned error can't find the container with id 55e3070ba193b5bf6a529fcad06bc5c997eb44d91d258edf28a977b39a99077a Jul 09 15:31:29.226441 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.226427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9hvvb" Jul 09 15:31:29.228740 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.228718 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d88c134_1d09_4f69_86a6_6d13c89a08b2.slice/crio-7422a8ecfa0b1ebce3bd9ee3af727c601f25a314d14294303c01bcd04ee64c71 WatchSource:0}: Error finding container 7422a8ecfa0b1ebce3bd9ee3af727c601f25a314d14294303c01bcd04ee64c71: Status 404 returned error can't find the container with id 7422a8ecfa0b1ebce3bd9ee3af727c601f25a314d14294303c01bcd04ee64c71 Jul 09 15:31:29.231327 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.231312 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" Jul 09 15:31:29.235281 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.235262 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa1ce07_ade0_407b_a5fa_9785a622f1b9.slice/crio-16128d75feaac22fb2a5152c254b72a62f8c8b19305900eabdca7fb68bd55c63 WatchSource:0}: Error finding container 16128d75feaac22fb2a5152c254b72a62f8c8b19305900eabdca7fb68bd55c63: Status 404 returned error can't find the container with id 16128d75feaac22fb2a5152c254b72a62f8c8b19305900eabdca7fb68bd55c63 Jul 09 15:31:29.236599 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.236581 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4zmmk" Jul 09 15:31:29.240141 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.240107 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaea8bbb_2a0c_49be_b64a_dbfd3fe9001c.slice/crio-b88a506629793203da80b12d6c62b90d2655b02039e1cb4ec5469017111442a2 WatchSource:0}: Error finding container b88a506629793203da80b12d6c62b90d2655b02039e1cb4ec5469017111442a2: Status 404 returned error can't find the container with id b88a506629793203da80b12d6c62b90d2655b02039e1cb4ec5469017111442a2 Jul 09 15:31:29.241692 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.241674 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cz222" Jul 09 15:31:29.246762 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.246747 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:29.251132 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.251112 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode873854a_76a3_4172_ae5e_bbd3af1a04b7.slice/crio-bac77c35fd0c943548707818916589f5eea4ae5e9d389fe1ee29b18df690b40a WatchSource:0}: Error finding container bac77c35fd0c943548707818916589f5eea4ae5e9d389fe1ee29b18df690b40a: Status 404 returned error can't find the container with id bac77c35fd0c943548707818916589f5eea4ae5e9d389fe1ee29b18df690b40a Jul 09 15:31:29.252100 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.252084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-np5vh" Jul 09 15:31:29.254487 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.254449 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547d60d5_feb0_45c1_af93_9eb6d5b593c4.slice/crio-80cea9a101d45203cf7268042cce8fe16302f6744bc5476dfffa0e51f3b139a5 WatchSource:0}: Error finding container 80cea9a101d45203cf7268042cce8fe16302f6744bc5476dfffa0e51f3b139a5: Status 404 returned error can't find the container with id 80cea9a101d45203cf7268042cce8fe16302f6744bc5476dfffa0e51f3b139a5 Jul 09 15:31:29.256642 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.256630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:29.259582 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.259566 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32759466_2823_4980_a821_990d82f7b87b.slice/crio-de2b9ddc09978089dc50c1e2176a9343ec0a16beac76711f234f7fb6ee16f3e4 WatchSource:0}: Error finding container de2b9ddc09978089dc50c1e2176a9343ec0a16beac76711f234f7fb6ee16f3e4: Status 404 returned error can't find the container with id de2b9ddc09978089dc50c1e2176a9343ec0a16beac76711f234f7fb6ee16f3e4 Jul 09 15:31:29.264495 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:31:29.264472 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f176317_d27b_4f21_b235_359b94766bbd.slice/crio-05f17bcf7b8dc7b8b2d98971d107c7b350f30e46c74a764db45a6af70f82f4c1 WatchSource:0}: Error finding container 05f17bcf7b8dc7b8b2d98971d107c7b350f30e46c74a764db45a6af70f82f4c1: Status 404 returned error can't find the container with id 05f17bcf7b8dc7b8b2d98971d107c7b350f30e46c74a764db45a6af70f82f4c1 Jul 09 15:31:29.563012 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.562935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:29.563012 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.562997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:29.563188 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.563068 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:29.563188 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.563071 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:29.563188 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.563116 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:31:30.563103648 +0000 UTC m=+7.100330799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:29.563188 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.563138 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:31:30.563125823 +0000 UTC m=+7.100352979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:29.663581 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.663544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:29.663754 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.663725 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:31:29.663823 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.663754 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:31:29.663823 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.663770 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:29.663917 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:29.663842 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:31:30.663814592 +0000 UTC m=+7.201041751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:29.969978 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.969899 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 15:26:28 +0000 UTC" deadline="2028-01-09 13:24:59.292893758 +0000 UTC" Jul 09 15:31:29.969978 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:29.969928 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13173h53m29.322970529s" Jul 09 15:31:30.155948 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.155875 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9hvvb" event={"ID":"1aa1ce07-ade0-407b-a5fa-9785a622f1b9","Type":"ContainerStarted","Data":"16128d75feaac22fb2a5152c254b72a62f8c8b19305900eabdca7fb68bd55c63"} Jul 09 15:31:30.166477 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.166357 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-glxth" event={"ID":"27e8fecd-4af3-48b4-af25-7636492aac49","Type":"ContainerStarted","Data":"55e3070ba193b5bf6a529fcad06bc5c997eb44d91d258edf28a977b39a99077a"} Jul 09 15:31:30.169691 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.169659 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cz222" event={"ID":"e873854a-76a3-4172-ae5e-bbd3af1a04b7","Type":"ContainerStarted","Data":"bac77c35fd0c943548707818916589f5eea4ae5e9d389fe1ee29b18df690b40a"} Jul 09 15:31:30.171967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.171923 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4zmmk" event={"ID":"45f9fd98-9365-48d7-a594-1ca287fd2234","Type":"ContainerStarted","Data":"2a75d8693fb58d186fb6664374bb5dbba9320c84423b6ac1812b22ffeb8fc8ec"} Jul 09 15:31:30.179193 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.179166 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" event={"ID":"9d88c134-1d09-4f69-86a6-6d13c89a08b2","Type":"ContainerStarted","Data":"7422a8ecfa0b1ebce3bd9ee3af727c601f25a314d14294303c01bcd04ee64c71"} Jul 09 15:31:30.190703 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.190131 2577 generic.go:356] "Generic (PLEG): container finished" podID="07c2d5eaa8bf47df92ffe81e309b9062" containerID="b1655a385e874135d51cb112126db9633abaf0d62d7d964dfc1e04e112247847" exitCode=0 Jul 09 15:31:30.190703 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.190202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" event={"ID":"07c2d5eaa8bf47df92ffe81e309b9062","Type":"ContainerDied","Data":"b1655a385e874135d51cb112126db9633abaf0d62d7d964dfc1e04e112247847"} Jul 09 15:31:30.195359 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.195278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9m4mg" event={"ID":"5f176317-d27b-4f21-b235-359b94766bbd","Type":"ContainerStarted","Data":"05f17bcf7b8dc7b8b2d98971d107c7b350f30e46c74a764db45a6af70f82f4c1"} Jul 09 15:31:30.199044 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.198996 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerStarted","Data":"de2b9ddc09978089dc50c1e2176a9343ec0a16beac76711f234f7fb6ee16f3e4"} Jul 09 15:31:30.209717 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.209615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"80cea9a101d45203cf7268042cce8fe16302f6744bc5476dfffa0e51f3b139a5"} Jul 09 15:31:30.216865 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.216839 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" event={"ID":"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c","Type":"ContainerStarted","Data":"b88a506629793203da80b12d6c62b90d2655b02039e1cb4ec5469017111442a2"} Jul 09 15:31:30.577514 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.577403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:30.577514 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.577474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:30.577756 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.577641 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:30.577756 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.577719 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:31:32.577700788 +0000 UTC m=+9.114927943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:30.578122 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.578107 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:30.578203 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.578155 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:31:32.578141647 +0000 UTC m=+9.115368803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:30.678270 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.678232 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:30.678487 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.678440 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:31:30.678487 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.678481 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:31:30.678600 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.678494 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:30.678600 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:30.678554 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:31:32.678534906 +0000 UTC m=+9.215762079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:30.970951 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.970886 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 15:26:28 +0000 UTC" deadline="2028-03-02 20:30:21.290773867 +0000 UTC" Jul 09 15:31:30.970951 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:30.970949 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14452h58m50.319828979s" Jul 09 15:31:31.130572 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:31.130535 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:31.130736 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:31.130678 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:31.131112 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:31.131091 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:31.131202 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:31.131188 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:31.131265 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:31.131249 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:31.131357 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:31.131332 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:31.236039 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:31.235959 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" event={"ID":"07c2d5eaa8bf47df92ffe81e309b9062","Type":"ContainerStarted","Data":"c351b5f187667211a60635deb157369d53dc2a582f5162dd1a855358cd0f216f"} Jul 09 15:31:32.595886 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:32.595855 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:32.596516 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:32.595900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:32.596516 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.595999 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:32.596516 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.596028 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:32.596516 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.596061 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:31:36.596041751 +0000 UTC m=+13.133268916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:32.596516 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.596078 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:31:36.596069598 +0000 UTC m=+13.133296749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:32.697005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:32.696730 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:32.697005 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.696937 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:31:32.697005 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.696958 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:31:32.697005 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.696970 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:32.697356 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:32.697021 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:31:36.697003568 +0000 UTC m=+13.234230733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:33.130070 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:33.130035 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:33.130258 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:33.130170 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:33.130589 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:33.130569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:33.130708 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:33.130668 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:33.130790 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:33.130771 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:33.131003 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:33.130894 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:35.130254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:35.130210 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:35.130254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:35.130256 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:35.130808 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:35.130210 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:35.130808 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:35.130351 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:35.130808 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:35.130474 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:35.130808 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:35.130566 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:36.630933 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:36.630893 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:36.631385 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:36.630940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:36.631385 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.631063 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:36.631385 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.631140 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:31:44.631121611 +0000 UTC m=+21.168348768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:36.631385 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.631063 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:36.631385 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.631192 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:31:44.631180947 +0000 UTC m=+21.168408104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:36.731734 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:36.731695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:36.732345 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.731903 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:31:36.732345 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.731936 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:31:36.732345 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.731950 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:36.732345 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:36.732010 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:31:44.731989726 +0000 UTC m=+21.269216895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:37.129884 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:37.129852 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:37.129884 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:37.129874 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:37.129884 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:37.129865 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:37.130179 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:37.130011 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:37.130179 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:37.130056 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:37.130179 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:37.130150 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:39.130533 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:39.130497 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:39.131006 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:39.130628 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:39.131006 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:39.130655 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:39.131006 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:39.130656 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:39.131006 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:39.130766 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:39.131006 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:39.130829 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:40.262922 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.262698 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" event={"ID":"caea8bbb-2a0c-49be-b64a-dbfd3fe9001c","Type":"ContainerStarted","Data":"515b902f9220418e0d465be14b88dd23d16c8363453e95c7190f9b699c841ab3"} Jul 09 15:31:40.264314 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.264274 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cz222" event={"ID":"e873854a-76a3-4172-ae5e-bbd3af1a04b7","Type":"ContainerStarted","Data":"3e7619afe1a5cb675a7ab7f887a72dd81b0efdd072370b0555f57c10b02fe398"} Jul 09 15:31:40.265881 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.265853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4zmmk" event={"ID":"45f9fd98-9365-48d7-a594-1ca287fd2234","Type":"ContainerStarted","Data":"977ad5150394a5fb97145a87f88e4921fdfcf8c4f64537177f8dc4ce8ef876f1"} Jul 09 15:31:40.267418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.267389 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" event={"ID":"9d88c134-1d09-4f69-86a6-6d13c89a08b2","Type":"ContainerStarted","Data":"5a96574251902413b23b752557e10636228b81ee67d9adae785ad91a9f5e24a7"} Jul 09 15:31:40.268841 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.268818 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9m4mg" event={"ID":"5f176317-d27b-4f21-b235-359b94766bbd","Type":"ContainerStarted","Data":"5511289c46d58dc274298847084b3b549438b0ada2a1231693d11f58ff643be8"} Jul 09 15:31:40.270397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.270371 2577 generic.go:356] "Generic (PLEG): container finished" podID="32759466-2823-4980-a821-990d82f7b87b" containerID="4eedf9636579f3e59897926310511734777ee8d9ff90e2425bc5cf89ed3d7b6d" exitCode=0 Jul 09 15:31:40.270513 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.270407 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerDied","Data":"4eedf9636579f3e59897926310511734777ee8d9ff90e2425bc5cf89ed3d7b6d"} Jul 09 15:31:40.281234 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.281173 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4pfnr" podStartSLOduration=3.334033843 podStartE2EDuration="13.281155537s" totalImagesPullingTime="9.947121694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:40.280733937 +0000 UTC m=+16.817961110" watchObservedRunningTime="2026-07-09 15:31:40.281155537 +0000 UTC m=+16.818382711" Jul 09 15:31:40.281441 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.281407 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-0.ec2.internal" podStartSLOduration=12.281398497 podStartE2EDuration="12.281398497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:31.254847253 +0000 UTC m=+7.792074428" watchObservedRunningTime="2026-07-09 15:31:40.281398497 +0000 UTC m=+16.818625670" Jul 09 15:31:40.317072 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.316991 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4zmmk" podStartSLOduration=3.394443437 podStartE2EDuration="13.316975807s" totalImagesPullingTime="9.92253237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:40.316860077 +0000 UTC m=+16.854087252" watchObservedRunningTime="2026-07-09 15:31:40.316975807 +0000 UTC m=+16.854202981" Jul 09 15:31:40.335733 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.335690 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cz222" podStartSLOduration=3.419366892 podStartE2EDuration="13.335676048s" totalImagesPullingTime="9.916309156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:40.335580361 +0000 UTC m=+16.872807534" watchObservedRunningTime="2026-07-09 15:31:40.335676048 +0000 UTC m=+16.872903220" Jul 09 15:31:40.430960 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:40.430921 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:41.130622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:41.130584 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:41.130622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:41.130611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:41.130871 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:41.130611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:41.130871 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:41.130725 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:41.130871 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:41.130813 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:41.131009 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:41.130927 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:41.273834 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:41.273789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9hvvb" event={"ID":"1aa1ce07-ade0-407b-a5fa-9785a622f1b9","Type":"ContainerStarted","Data":"f941969021b87a557341e6bbc71a11cc7a4cdf07b84b53d130deb6b3780c3fe9"} Jul 09 15:31:41.290781 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:41.290726 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9hvvb" podStartSLOduration=4.358451197 podStartE2EDuration="14.290709369s" totalImagesPullingTime="9.932258172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:41.290493483 +0000 UTC m=+17.827720659" watchObservedRunningTime="2026-07-09 15:31:41.290709369 +0000 UTC m=+17.827936542" Jul 09 15:31:41.291328 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:41.291294 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9m4mg" podStartSLOduration=4.409535544 podStartE2EDuration="14.291285138s" totalImagesPullingTime="9.881749594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:40.349290935 +0000 UTC m=+16.886518107" watchObservedRunningTime="2026-07-09 15:31:41.291285138 +0000 UTC m=+17.828512680" Jul 09 15:31:43.130182 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:43.130147 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:43.130182 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:43.130188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:43.130763 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:43.130149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:43.130763 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:43.130289 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:43.130763 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:43.130334 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:43.130763 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:43.130394 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:44.257300 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:44.257263 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:44.257981 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:44.257964 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:44.279694 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:44.279655 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9m4mg" Jul 09 15:31:44.692768 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:44.692726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:44.692947 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:44.692780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:44.692947 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.692906 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:44.692947 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.692911 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:44.693110 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.692973 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:32:00.692955176 +0000 UTC m=+37.230182345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:31:44.693110 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.692992 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:32:00.692982983 +0000 UTC m=+37.230210141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:31:44.793713 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:44.793616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:44.793898 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.793767 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:31:44.793898 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.793790 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:31:44.793898 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.793801 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:44.793898 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:44.793862 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:32:00.793844576 +0000 UTC m=+37.331071740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:31:45.130239 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:45.130206 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:45.130239 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:45.130228 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:45.130492 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:45.130255 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:45.130492 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:45.130387 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:45.130492 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:45.130400 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:45.130656 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:45.130506 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:47.130483 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:47.130416 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:47.131141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:47.130588 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:47.131141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:47.130606 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:47.131141 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:47.130599 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:47.131141 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:47.130698 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:47.131141 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:47.130770 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:49.130276 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:49.130238 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:49.130276 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:49.130266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:49.130808 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:49.130238 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:49.130808 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:49.130371 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:49.130808 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:49.130490 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:49.130808 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:49.130604 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:49.905675 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:49.905391 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 15:31:50.013612 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.013507 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T15:31:49.905672629Z","UUID":"c5c69bb3-564b-4902-bef5-09ef074f74a9","Handler":null,"Name":"","Endpoint":""} Jul 09 15:31:50.016770 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.016746 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 15:31:50.016900 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.016796 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 15:31:50.293187 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.292919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-glxth" event={"ID":"27e8fecd-4af3-48b4-af25-7636492aac49","Type":"ContainerStarted","Data":"a1e014398db186ee748a831e19d7bde06c96d891d78df74aed02e7c69c8b5f33"} Jul 09 15:31:50.294743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.294716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" event={"ID":"9d88c134-1d09-4f69-86a6-6d13c89a08b2","Type":"ContainerStarted","Data":"05fa43b67201b29d8a250b1191d10a7233efa729fd7cecafd47feae7f13ccaf8"} Jul 09 15:31:50.296403 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.296383 2577 generic.go:356] "Generic (PLEG): container finished" podID="32759466-2823-4980-a821-990d82f7b87b" containerID="7a33c5de8b09b7a41d4b59a0774dc76ce4b402bee6a6cf9a6d66acccbb45c6ac" exitCode=0 Jul 09 15:31:50.296502 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.296450 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerDied","Data":"7a33c5de8b09b7a41d4b59a0774dc76ce4b402bee6a6cf9a6d66acccbb45c6ac"} Jul 09 15:31:50.299028 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299011 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:31:50.299340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299323 2577 generic.go:356] "Generic (PLEG): container finished" podID="547d60d5-feb0-45c1-af93-9eb6d5b593c4" containerID="fb61b4a8b822c0002d89e97c07cfa229a7b6636c34743f42f647c7eb2a6c307b" exitCode=1 Jul 09 15:31:50.299394 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299352 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"2f43e3427297ac97c7a5cc1c8de54c47a1a363ff0ea3f3299fb1520f73d227ca"} Jul 09 15:31:50.299432 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"188cccba9147fbb9240cb26c0d7bb66238b19f1407dae4e893803ff92196121b"} Jul 09 15:31:50.299500 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299485 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"b487346df71792cacc7eb8f4b76d56d94ea4401483d8c63a7895362ed84b3e42"} Jul 09 15:31:50.299555 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299542 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerDied","Data":"fb61b4a8b822c0002d89e97c07cfa229a7b6636c34743f42f647c7eb2a6c307b"} Jul 09 15:31:50.299609 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.299597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"30848eb51e2394319c3955c013b9c7e8282338a0cedda1f7454db9d440902c29"} Jul 09 15:31:50.315021 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:50.314982 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-glxth" podStartSLOduration=2.7569889180000002 podStartE2EDuration="23.314966555s" totalImagesPullingTime="20.557977637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:50.31485796 +0000 UTC m=+26.852085134" watchObservedRunningTime="2026-07-09 15:31:50.314966555 +0000 UTC m=+26.852193728" Jul 09 15:31:51.129969 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:51.129942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:51.130127 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:51.129973 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:51.130127 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:51.129936 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:51.130127 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:51.130056 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:51.130241 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:51.130124 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:51.130241 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:51.130188 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:51.304574 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:51.304550 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:31:51.304965 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:51.304944 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"a377733515c359d2c03d2f2092382bc8b4bd610b08cade8156e08c0baf0412ea"} Jul 09 15:31:52.309321 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:52.309278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" event={"ID":"9d88c134-1d09-4f69-86a6-6d13c89a08b2","Type":"ContainerStarted","Data":"f6f0a4f9e479fd091777afd48ac33b34e862c7f3cf550d3b3e7f75a7320c0918"} Jul 09 15:31:52.311350 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:52.311326 2577 generic.go:356] "Generic (PLEG): container finished" podID="32759466-2823-4980-a821-990d82f7b87b" containerID="b0570c744f4080545b3562e6a392df14c1f860de49da939b7e77b32ccce25e94" exitCode=0 Jul 09 15:31:52.311492 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:52.311367 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerDied","Data":"b0570c744f4080545b3562e6a392df14c1f860de49da939b7e77b32ccce25e94"} Jul 09 15:31:52.327916 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:52.327874 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8kll" podStartSLOduration=3.388029694 podStartE2EDuration="25.327861129s" totalImagesPullingTime="21.939831435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:52.32745009 +0000 UTC m=+28.864677265" watchObservedRunningTime="2026-07-09 15:31:52.327861129 +0000 UTC m=+28.865088303" Jul 09 15:31:53.130160 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:53.130124 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:53.130353 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:53.130124 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:53.130353 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:53.130265 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:53.130353 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:53.130124 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:53.130353 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:53.130315 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:53.130538 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:53.130388 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:53.316825 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:53.316798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:31:53.317306 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:53.317193 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"bdf0b652c57149f2828b288531daabc0bffdbe406f9de4cc72b2bedb5a7f78dc"} Jul 09 15:31:54.321637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:54.321594 2577 generic.go:356] "Generic (PLEG): container finished" podID="32759466-2823-4980-a821-990d82f7b87b" containerID="7235492f14f3f49874beb1eade457e87bd904103f8e7d2e21720501b95b5c31d" exitCode=0 Jul 09 15:31:54.322006 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:54.321657 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerDied","Data":"7235492f14f3f49874beb1eade457e87bd904103f8e7d2e21720501b95b5c31d"} Jul 09 15:31:55.130635 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.130598 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:55.130805 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.130598 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:55.130805 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:55.130721 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:55.130805 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.130605 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:55.130805 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:55.130794 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:55.131026 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:55.130919 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:55.328264 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.328234 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:31:55.328795 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.328767 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"b5aa71ce10a4d5aea44da2c5d76105697468f9bdab04ddd74cf2ada9d519a617"} Jul 09 15:31:55.329273 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.329115 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:55.329273 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.329198 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:55.329423 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.329381 2577 scope.go:117] "RemoveContainer" containerID="fb61b4a8b822c0002d89e97c07cfa229a7b6636c34743f42f647c7eb2a6c307b" Jul 09 15:31:55.346665 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:55.346400 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:56.336711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.336682 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:31:56.337151 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.337100 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" event={"ID":"547d60d5-feb0-45c1-af93-9eb6d5b593c4","Type":"ContainerStarted","Data":"ac5807ecbb9848e1e1d0d1fe5f24478f73b2ac7966b1d91378a1417c167245c5"} Jul 09 15:31:56.337357 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.337333 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:56.352654 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.352629 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:31:56.369303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.369251 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" podStartSLOduration=8.836275023 podStartE2EDuration="29.36923374s" totalImagesPullingTime="20.532958717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:56.368308759 +0000 UTC m=+32.905535934" watchObservedRunningTime="2026-07-09 15:31:56.36923374 +0000 UTC m=+32.906460917" Jul 09 15:31:56.992449 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.992404 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wx6lp"] Jul 09 15:31:56.992644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.992592 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:56.992725 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:56.992697 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:56.994255 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.994230 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7k7t"] Jul 09 15:31:56.994418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.994346 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:56.994535 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:56.994455 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:56.994813 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.994794 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8nz45"] Jul 09 15:31:56.994909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:56.994875 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:56.994993 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:56.994950 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:59.129982 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:59.129940 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:31:59.129982 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:59.129974 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:31:59.130741 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:59.130050 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:31:59.130741 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:59.129958 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:31:59.130741 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:59.130158 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:31:59.130741 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:31:59.130280 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:31:59.346644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:59.346565 2577 generic.go:356] "Generic (PLEG): container finished" podID="32759466-2823-4980-a821-990d82f7b87b" containerID="cce53c6c7457a9a6af07f520259c0d5ac60fc298a5b19e2070df5351143682c0" exitCode=0 Jul 09 15:31:59.346644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:31:59.346620 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerDied","Data":"cce53c6c7457a9a6af07f520259c0d5ac60fc298a5b19e2070df5351143682c0"} Jul 09 15:32:00.352108 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:00.351828 2577 generic.go:356] "Generic (PLEG): container finished" podID="32759466-2823-4980-a821-990d82f7b87b" containerID="d9e714046514a3a5f8a8ef005f86482cc24cece81b1a064e817a42ce2b894278" exitCode=0 Jul 09 15:32:00.352524 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:00.351923 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerDied","Data":"d9e714046514a3a5f8a8ef005f86482cc24cece81b1a064e817a42ce2b894278"} Jul 09 15:32:00.708589 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:00.708552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:32:00.708767 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:00.708650 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:32:00.708767 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.708701 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:32:00.708767 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.708726 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 15:32:00.708921 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.708769 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret podName:b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d nodeName:}" failed. No retries permitted until 2026-07-09 15:32:32.708753194 +0000 UTC m=+69.245980346 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret") pod "global-pull-secret-syncer-8nz45" (UID: "b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d") : object "kube-system"/"original-pull-secret" not registered Jul 09 15:32:00.708921 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.708787 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs podName:1c3ed855-db93-4389-b437-eb12d222712b nodeName:}" failed. No retries permitted until 2026-07-09 15:32:32.708778786 +0000 UTC m=+69.246005939 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs") pod "network-metrics-daemon-s7k7t" (UID: "1c3ed855-db93-4389-b437-eb12d222712b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 15:32:00.809562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:00.809521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:00.809738 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.809704 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 15:32:00.809738 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.809724 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 15:32:00.809738 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.809735 2577 projected.go:196] Error preparing data for projected volume kube-api-access-dvpvz for pod openshift-network-diagnostics/network-check-target-wx6lp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:32:00.809837 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:00.809786 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz podName:207bf408-a81a-47e7-8f81-bf939ad0227a nodeName:}" failed. No retries permitted until 2026-07-09 15:32:32.809772279 +0000 UTC m=+69.346999432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dvpvz" (UniqueName: "kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz") pod "network-check-target-wx6lp" (UID: "207bf408-a81a-47e7-8f81-bf939ad0227a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 15:32:01.130526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:01.130493 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:32:01.130691 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:01.130514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:32:01.130691 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:01.130614 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8nz45" podUID="b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d" Jul 09 15:32:01.130691 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:01.130620 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:01.130798 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:01.130695 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7k7t" podUID="1c3ed855-db93-4389-b437-eb12d222712b" Jul 09 15:32:01.130798 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:01.130760 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wx6lp" podUID="207bf408-a81a-47e7-8f81-bf939ad0227a" Jul 09 15:32:01.357246 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:01.357201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-np5vh" event={"ID":"32759466-2823-4980-a821-990d82f7b87b","Type":"ContainerStarted","Data":"4a98e0036c38eee9c56f6bbf3091068c20dc61fcb1a741e472fd73a7217c1f2f"} Jul 09 15:32:01.382906 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:01.382810 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-np5vh" podStartSLOduration=7.387188788 podStartE2EDuration="34.382794721s" totalImagesPullingTime="26.995605933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:01.382484666 +0000 UTC m=+37.919711834" watchObservedRunningTime="2026-07-09 15:32:01.382794721 +0000 UTC m=+37.920021894" Jul 09 15:32:02.279080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.279049 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-0.ec2.internal" event="NodeReady" Jul 09 15:32:02.279245 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.279184 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 15:32:02.330019 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.329991 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jvpzg"] Jul 09 15:32:02.348219 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.348183 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5lgqr"] Jul 09 15:32:02.348397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.348310 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.350665 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.350641 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 15:32:02.350797 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.350679 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 15:32:02.350797 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.350705 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zcxss\"" Jul 09 15:32:02.368283 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.368253 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jvpzg"] Jul 09 15:32:02.368283 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.368289 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5lgqr"] Jul 09 15:32:02.368760 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.368384 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.370918 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.370890 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 15:32:02.371048 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.370957 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 15:32:02.371048 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.370961 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 15:32:02.371230 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.371218 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wrjlr\"" Jul 09 15:32:02.521733 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.521636 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f46785e3-38b5-4c15-8891-270e1983b38e-tmp-dir\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.521733 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.521681 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9khzq\" (UniqueName: \"kubernetes.io/projected/f46785e3-38b5-4c15-8891-270e1983b38e-kube-api-access-9khzq\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.521733 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.521703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7pp9\" (UniqueName: \"kubernetes.io/projected/200101ee-c3b4-49b8-93da-a6cf99fa6061-kube-api-access-q7pp9\") pod \"ingress-canary-5lgqr\" (UID: \"200101ee-c3b4-49b8-93da-a6cf99fa6061\") " pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.522016 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.521897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f46785e3-38b5-4c15-8891-270e1983b38e-config-volume\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.522016 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.521939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.522127 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.522033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/200101ee-c3b4-49b8-93da-a6cf99fa6061-cert\") pod \"ingress-canary-5lgqr\" (UID: \"200101ee-c3b4-49b8-93da-a6cf99fa6061\") " pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.622776 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.622739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f46785e3-38b5-4c15-8891-270e1983b38e-tmp-dir\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.622776 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.622781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9khzq\" (UniqueName: \"kubernetes.io/projected/f46785e3-38b5-4c15-8891-270e1983b38e-kube-api-access-9khzq\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.623047 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.622800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7pp9\" (UniqueName: \"kubernetes.io/projected/200101ee-c3b4-49b8-93da-a6cf99fa6061-kube-api-access-q7pp9\") pod \"ingress-canary-5lgqr\" (UID: \"200101ee-c3b4-49b8-93da-a6cf99fa6061\") " pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.623047 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.622849 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f46785e3-38b5-4c15-8891-270e1983b38e-config-volume\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.623047 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.622871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.623047 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.622891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/200101ee-c3b4-49b8-93da-a6cf99fa6061-cert\") pod \"ingress-canary-5lgqr\" (UID: \"200101ee-c3b4-49b8-93da-a6cf99fa6061\") " pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.623047 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:02.622988 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 15:32:02.623268 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:02.623093 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls podName:f46785e3-38b5-4c15-8891-270e1983b38e nodeName:}" failed. No retries permitted until 2026-07-09 15:32:03.123072182 +0000 UTC m=+39.660299338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls") pod "dns-default-jvpzg" (UID: "f46785e3-38b5-4c15-8891-270e1983b38e") : secret "dns-default-metrics-tls" not found Jul 09 15:32:02.623268 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.623175 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f46785e3-38b5-4c15-8891-270e1983b38e-tmp-dir\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.623431 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.623410 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f46785e3-38b5-4c15-8891-270e1983b38e-config-volume\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.627125 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.627107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/200101ee-c3b4-49b8-93da-a6cf99fa6061-cert\") pod \"ingress-canary-5lgqr\" (UID: \"200101ee-c3b4-49b8-93da-a6cf99fa6061\") " pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.632031 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.632005 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7pp9\" (UniqueName: \"kubernetes.io/projected/200101ee-c3b4-49b8-93da-a6cf99fa6061-kube-api-access-q7pp9\") pod \"ingress-canary-5lgqr\" (UID: \"200101ee-c3b4-49b8-93da-a6cf99fa6061\") " pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.632198 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.632179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9khzq\" (UniqueName: \"kubernetes.io/projected/f46785e3-38b5-4c15-8891-270e1983b38e-kube-api-access-9khzq\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:02.676262 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.676223 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5lgqr" Jul 09 15:32:02.857884 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:02.857853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5lgqr"] Jul 09 15:32:02.864916 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:02.864880 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200101ee_c3b4_49b8_93da_a6cf99fa6061.slice/crio-fdabb3e8e12332a709a9426159c641a8f7bcabb7092203463b809b856782b820 WatchSource:0}: Error finding container fdabb3e8e12332a709a9426159c641a8f7bcabb7092203463b809b856782b820: Status 404 returned error can't find the container with id fdabb3e8e12332a709a9426159c641a8f7bcabb7092203463b809b856782b820 Jul 09 15:32:03.125922 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.125892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:03.126106 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:03.126071 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 15:32:03.126177 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:03.126156 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls podName:f46785e3-38b5-4c15-8891-270e1983b38e nodeName:}" failed. No retries permitted until 2026-07-09 15:32:04.126131747 +0000 UTC m=+40.663358919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls") pod "dns-default-jvpzg" (UID: "f46785e3-38b5-4c15-8891-270e1983b38e") : secret "dns-default-metrics-tls" not found Jul 09 15:32:03.129952 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.129925 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:03.130055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.130023 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:32:03.130125 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.130109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:32:03.132604 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.132575 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 15:32:03.132701 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.132609 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 15:32:03.132995 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.132981 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnkz7\"" Jul 09 15:32:03.133218 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.133201 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 15:32:03.133290 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.133246 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 15:32:03.133290 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.133261 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rsq6f\"" Jul 09 15:32:03.307005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.306977 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-gtpq2"] Jul 09 15:32:03.337899 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.337859 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-gtpq2"] Jul 09 15:32:03.338075 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.337956 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.340631 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.340610 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 15:32:03.340748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.340664 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 15:32:03.340748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.340615 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 15:32:03.340748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.340714 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kbhhw\"" Jul 09 15:32:03.340904 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.340753 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 15:32:03.362931 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.362895 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5lgqr" event={"ID":"200101ee-c3b4-49b8-93da-a6cf99fa6061","Type":"ContainerStarted","Data":"fdabb3e8e12332a709a9426159c641a8f7bcabb7092203463b809b856782b820"} Jul 09 15:32:03.428483 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.428377 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5289c4e4-bf4d-4f65-91d8-7070841f457d-signing-key\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.428483 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.428430 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5289c4e4-bf4d-4f65-91d8-7070841f457d-signing-cabundle\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.428856 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.428559 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bnl7\" (UniqueName: \"kubernetes.io/projected/5289c4e4-bf4d-4f65-91d8-7070841f457d-kube-api-access-6bnl7\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.529080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.529044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5289c4e4-bf4d-4f65-91d8-7070841f457d-signing-cabundle\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.529241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.529106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bnl7\" (UniqueName: \"kubernetes.io/projected/5289c4e4-bf4d-4f65-91d8-7070841f457d-kube-api-access-6bnl7\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.529241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.529144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5289c4e4-bf4d-4f65-91d8-7070841f457d-signing-key\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.529725 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.529704 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5289c4e4-bf4d-4f65-91d8-7070841f457d-signing-cabundle\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.531949 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.531932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5289c4e4-bf4d-4f65-91d8-7070841f457d-signing-key\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.538038 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.538013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bnl7\" (UniqueName: \"kubernetes.io/projected/5289c4e4-bf4d-4f65-91d8-7070841f457d-kube-api-access-6bnl7\") pod \"service-ca-79f68b9f4f-gtpq2\" (UID: \"5289c4e4-bf4d-4f65-91d8-7070841f457d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.646612 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.646574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" Jul 09 15:32:03.806819 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:03.806786 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-gtpq2"] Jul 09 15:32:03.810474 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:03.810433 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289c4e4_bf4d_4f65_91d8_7070841f457d.slice/crio-36cec85442fa7aca38cbb81f913221ba501541d41e8a9f47c71b5b0ff0c11464 WatchSource:0}: Error finding container 36cec85442fa7aca38cbb81f913221ba501541d41e8a9f47c71b5b0ff0c11464: Status 404 returned error can't find the container with id 36cec85442fa7aca38cbb81f913221ba501541d41e8a9f47c71b5b0ff0c11464 Jul 09 15:32:04.134354 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:04.134309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:04.134554 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:04.134443 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 15:32:04.134554 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:04.134521 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls podName:f46785e3-38b5-4c15-8891-270e1983b38e nodeName:}" failed. No retries permitted until 2026-07-09 15:32:06.134505865 +0000 UTC m=+42.671733016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls") pod "dns-default-jvpzg" (UID: "f46785e3-38b5-4c15-8891-270e1983b38e") : secret "dns-default-metrics-tls" not found Jul 09 15:32:04.366994 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:04.366954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" event={"ID":"5289c4e4-bf4d-4f65-91d8-7070841f457d","Type":"ContainerStarted","Data":"36cec85442fa7aca38cbb81f913221ba501541d41e8a9f47c71b5b0ff0c11464"} Jul 09 15:32:06.152682 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:06.152647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:06.153582 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:06.152789 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 15:32:06.153582 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:06.152846 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls podName:f46785e3-38b5-4c15-8891-270e1983b38e nodeName:}" failed. No retries permitted until 2026-07-09 15:32:10.152832439 +0000 UTC m=+46.690059591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls") pod "dns-default-jvpzg" (UID: "f46785e3-38b5-4c15-8891-270e1983b38e") : secret "dns-default-metrics-tls" not found Jul 09 15:32:06.373786 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:06.373563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5lgqr" event={"ID":"200101ee-c3b4-49b8-93da-a6cf99fa6061","Type":"ContainerStarted","Data":"44d9dfd50f571ff6c4b29e73d13dec5efb53e765df940d4ed26204cd690b28e8"} Jul 09 15:32:07.379154 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:07.379119 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" event={"ID":"5289c4e4-bf4d-4f65-91d8-7070841f457d","Type":"ContainerStarted","Data":"149e160c7a30d0976ea774b827bffa6d6fb68994b0566bca5941658a16cd6322"} Jul 09 15:32:07.395081 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:07.395027 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5lgqr" podStartSLOduration=2.396936069 podStartE2EDuration="5.394994019s" totalImagesPullingTime="2.99805795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:06.388836855 +0000 UTC m=+42.926064032" watchObservedRunningTime="2026-07-09 15:32:07.394994019 +0000 UTC m=+43.932221193" Jul 09 15:32:07.395265 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:07.395207 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-gtpq2" podStartSLOduration=1.001179735 podStartE2EDuration="4.395202511s" totalImagesPullingTime="3.394022776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:07.394651205 +0000 UTC m=+43.931878379" watchObservedRunningTime="2026-07-09 15:32:07.395202511 +0000 UTC m=+43.932429662" Jul 09 15:32:10.185905 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:10.185858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:10.186295 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:10.186015 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 15:32:10.186295 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:10.186072 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls podName:f46785e3-38b5-4c15-8891-270e1983b38e nodeName:}" failed. No retries permitted until 2026-07-09 15:32:18.186056145 +0000 UTC m=+54.723283300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls") pod "dns-default-jvpzg" (UID: "f46785e3-38b5-4c15-8891-270e1983b38e") : secret "dns-default-metrics-tls" not found Jul 09 15:32:18.247999 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:18.247955 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:18.251429 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:18.251385 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f46785e3-38b5-4c15-8891-270e1983b38e-metrics-tls\") pod \"dns-default-jvpzg\" (UID: \"f46785e3-38b5-4c15-8891-270e1983b38e\") " pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:18.257319 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:18.257279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:18.396933 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:18.396901 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jvpzg"] Jul 09 15:32:18.401051 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:18.401023 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46785e3_38b5_4c15_8891_270e1983b38e.slice/crio-89b656d352bbb1144e0bf6b422c6f47c6a7e0990d628b24e0ce489b73bd6e7ae WatchSource:0}: Error finding container 89b656d352bbb1144e0bf6b422c6f47c6a7e0990d628b24e0ce489b73bd6e7ae: Status 404 returned error can't find the container with id 89b656d352bbb1144e0bf6b422c6f47c6a7e0990d628b24e0ce489b73bd6e7ae Jul 09 15:32:18.407622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:18.407584 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvpzg" event={"ID":"f46785e3-38b5-4c15-8891-270e1983b38e","Type":"ContainerStarted","Data":"89b656d352bbb1144e0bf6b422c6f47c6a7e0990d628b24e0ce489b73bd6e7ae"} Jul 09 15:32:21.418141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:21.418098 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvpzg" event={"ID":"f46785e3-38b5-4c15-8891-270e1983b38e","Type":"ContainerStarted","Data":"204f896cd3314096e549d35d8c4e95fa3989e4cf04d8eecbdcd278f8604808ab"} Jul 09 15:32:21.418549 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:21.418186 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvpzg" event={"ID":"f46785e3-38b5-4c15-8891-270e1983b38e","Type":"ContainerStarted","Data":"cf6ba4cdf9ef0411d53aaea52f2f823a862d68b3bddfd81355f6a74f25b84a15"} Jul 09 15:32:21.418549 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:21.418243 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:21.439167 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:21.439000 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jvpzg" podStartSLOduration=16.928852503999998 podStartE2EDuration="19.438980052s" totalImagesPullingTime="2.510127548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:21.437984644 +0000 UTC m=+57.975211817" watchObservedRunningTime="2026-07-09 15:32:21.438980052 +0000 UTC m=+57.976207204" Jul 09 15:32:27.839960 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.839916 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp"] Jul 09 15:32:27.844259 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.844234 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm"] Jul 09 15:32:27.844404 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.844367 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:27.846296 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.846274 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" Jul 09 15:32:27.848685 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.848663 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:32:27.849126 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.849106 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 15:32:27.849233 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.849149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 15:32:27.849585 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.849545 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 15:32:27.849707 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.849635 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:32:27.849707 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.849649 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cpdln\"" Jul 09 15:32:27.849820 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.849656 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g67vz\"" Jul 09 15:32:27.852629 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.852611 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6dc6bcc468-7nttr"] Jul 09 15:32:27.856535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.855490 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:27.859487 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.857377 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp"] Jul 09 15:32:27.860573 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.860283 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 15:32:27.860682 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.860571 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vndc8\"" Jul 09 15:32:27.860754 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.860694 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 15:32:27.860810 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.860795 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 15:32:27.860991 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.860972 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 15:32:27.861297 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.861272 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 15:32:27.861383 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.861262 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 15:32:27.863394 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.863371 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm"] Jul 09 15:32:27.875630 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.875594 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dc6bcc468-7nttr"] Jul 09 15:32:27.921276 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921232 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-stats-auth\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:27.921276 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndzhz\" (UniqueName: \"kubernetes.io/projected/08707e88-88a7-46f7-b9c7-80008e16b603-kube-api-access-ndzhz\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:27.921562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921297 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-metrics-certs\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:27.921562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwbsk\" (UniqueName: \"kubernetes.io/projected/17ce1faa-e7ec-48e1-9563-2af943642e22-kube-api-access-pwbsk\") pod \"cluster-samples-operator-65db858894-psbpp\" (UID: \"17ce1faa-e7ec-48e1-9563-2af943642e22\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:27.921562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-default-certificate\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:27.921562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/17ce1faa-e7ec-48e1-9563-2af943642e22-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-psbpp\" (UID: \"17ce1faa-e7ec-48e1-9563-2af943642e22\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:27.921562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r94k6\" (UniqueName: \"kubernetes.io/projected/ea683fb5-2da7-4b3c-a8d9-42cfbfd03040-kube-api-access-r94k6\") pod \"volume-data-source-validator-78495597f6-nszjm\" (UID: \"ea683fb5-2da7-4b3c-a8d9-42cfbfd03040\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" Jul 09 15:32:27.921562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.921547 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08707e88-88a7-46f7-b9c7-80008e16b603-service-ca-bundle\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:27.941763 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.941729 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-w2vmq"] Jul 09 15:32:27.944135 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.944103 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns"] Jul 09 15:32:27.944286 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.944228 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:27.946156 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.946136 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d9c667bf6-bfgbb"] Jul 09 15:32:27.946304 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.946286 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:27.946981 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.946959 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 15:32:27.947105 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.946965 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 15:32:27.947369 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.947353 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-285tj\"" Jul 09 15:32:27.948543 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.948526 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-9xxq9"] Jul 09 15:32:27.948641 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.948557 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:27.949429 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.949408 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bz99d\"" Jul 09 15:32:27.949537 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.949517 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 15:32:27.949624 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.949543 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 15:32:27.949624 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.949610 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 15:32:27.949724 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.949611 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 15:32:27.950588 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.950569 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lq2hc\"" Jul 09 15:32:27.950685 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.950615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:27.950685 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.950617 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 15:32:27.950794 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.950772 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 15:32:27.951027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.951001 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 15:32:27.952977 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.952961 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9thk4\"" Jul 09 15:32:27.953238 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.953211 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 15:32:27.953496 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.953481 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 15:32:27.953759 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.953743 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 15:32:27.953832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.953775 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 15:32:27.957355 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.957334 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 15:32:27.958897 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.958878 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 15:32:27.960576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.960555 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns"] Jul 09 15:32:27.963578 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.963553 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-w2vmq"] Jul 09 15:32:27.966380 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.966350 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-9xxq9"] Jul 09 15:32:27.966511 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:27.966420 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d9c667bf6-bfgbb"] Jul 09 15:32:28.022066 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78b56a34-ea76-4944-99b5-11b561194852-registry-certificates\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022066 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-snapshots\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-registry-tls\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022099 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc6pl\" (UniqueName: \"kubernetes.io/projected/25ee3b41-af6d-4d49-a813-c9eee12fc550-kube-api-access-rc6pl\") pod \"downloads-68c9cbb497-w2vmq\" (UID: \"25ee3b41-af6d-4d49-a813-c9eee12fc550\") " pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9966p\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-kube-api-access-9966p\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-tmp\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022185 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndzhz\" (UniqueName: \"kubernetes.io/projected/08707e88-88a7-46f7-b9c7-80008e16b603-kube-api-access-ndzhz\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-stats-auth\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-metrics-certs\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb7qn\" (UniqueName: \"kubernetes.io/projected/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-kube-api-access-lb7qn\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.022278 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwbsk\" (UniqueName: \"kubernetes.io/projected/17ce1faa-e7ec-48e1-9563-2af943642e22-kube-api-access-pwbsk\") pod \"cluster-samples-operator-65db858894-psbpp\" (UID: \"17ce1faa-e7ec-48e1-9563-2af943642e22\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78b56a34-ea76-4944-99b5-11b561194852-trusted-ca\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-427b6\" (UniqueName: \"kubernetes.io/projected/55fb2e48-ed73-44be-bf45-c15342026d24-kube-api-access-427b6\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55fb2e48-ed73-44be-bf45-c15342026d24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-serving-cert\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r94k6\" (UniqueName: \"kubernetes.io/projected/ea683fb5-2da7-4b3c-a8d9-42cfbfd03040-kube-api-access-r94k6\") pod \"volume-data-source-validator-78495597f6-nszjm\" (UID: \"ea683fb5-2da7-4b3c-a8d9-42cfbfd03040\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08707e88-88a7-46f7-b9c7-80008e16b603-service-ca-bundle\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78b56a34-ea76-4944-99b5-11b561194852-ca-trust-extracted\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78b56a34-ea76-4944-99b5-11b561194852-installation-pull-secrets\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78b56a34-ea76-4944-99b5-11b561194852-image-registry-private-configuration\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-bound-sa-token\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.022829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/55fb2e48-ed73-44be-bf45-c15342026d24-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.023488 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022862 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-default-certificate\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.023488 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.022888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/17ce1faa-e7ec-48e1-9563-2af943642e22-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-psbpp\" (UID: \"17ce1faa-e7ec-48e1-9563-2af943642e22\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:28.023488 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.023351 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08707e88-88a7-46f7-b9c7-80008e16b603-service-ca-bundle\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.024980 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.024953 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-metrics-certs\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.024980 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.024969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-stats-auth\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.025137 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.025121 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/08707e88-88a7-46f7-b9c7-80008e16b603-default-certificate\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.025402 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.025383 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/17ce1faa-e7ec-48e1-9563-2af943642e22-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-psbpp\" (UID: \"17ce1faa-e7ec-48e1-9563-2af943642e22\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:28.034204 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.034181 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r94k6\" (UniqueName: \"kubernetes.io/projected/ea683fb5-2da7-4b3c-a8d9-42cfbfd03040-kube-api-access-r94k6\") pod \"volume-data-source-validator-78495597f6-nszjm\" (UID: \"ea683fb5-2da7-4b3c-a8d9-42cfbfd03040\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" Jul 09 15:32:28.034560 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.034535 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndzhz\" (UniqueName: \"kubernetes.io/projected/08707e88-88a7-46f7-b9c7-80008e16b603-kube-api-access-ndzhz\") pod \"router-default-6dc6bcc468-7nttr\" (UID: \"08707e88-88a7-46f7-b9c7-80008e16b603\") " pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.036173 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.036150 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwbsk\" (UniqueName: \"kubernetes.io/projected/17ce1faa-e7ec-48e1-9563-2af943642e22-kube-api-access-pwbsk\") pod \"cluster-samples-operator-65db858894-psbpp\" (UID: \"17ce1faa-e7ec-48e1-9563-2af943642e22\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:28.124168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-registry-tls\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124177 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc6pl\" (UniqueName: \"kubernetes.io/projected/25ee3b41-af6d-4d49-a813-c9eee12fc550-kube-api-access-rc6pl\") pod \"downloads-68c9cbb497-w2vmq\" (UID: \"25ee3b41-af6d-4d49-a813-c9eee12fc550\") " pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9966p\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-kube-api-access-9966p\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-tmp\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb7qn\" (UniqueName: \"kubernetes.io/projected/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-kube-api-access-lb7qn\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78b56a34-ea76-4944-99b5-11b561194852-trusted-ca\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-427b6\" (UniqueName: \"kubernetes.io/projected/55fb2e48-ed73-44be-bf45-c15342026d24-kube-api-access-427b6\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55fb2e48-ed73-44be-bf45-c15342026d24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.124340 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-serving-cert\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78b56a34-ea76-4944-99b5-11b561194852-ca-trust-extracted\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78b56a34-ea76-4944-99b5-11b561194852-installation-pull-secrets\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124556 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78b56a34-ea76-4944-99b5-11b561194852-image-registry-private-configuration\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-bound-sa-token\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124640 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/55fb2e48-ed73-44be-bf45-c15342026d24-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124673 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78b56a34-ea76-4944-99b5-11b561194852-registry-certificates\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124690 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-tmp\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.124783 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-snapshots\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.125344 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.124910 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78b56a34-ea76-4944-99b5-11b561194852-ca-trust-extracted\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.125344 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.125227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-snapshots\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.125624 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.125596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78b56a34-ea76-4944-99b5-11b561194852-trusted-ca\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.125806 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.125625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.126207 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.126182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.126299 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.126246 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/55fb2e48-ed73-44be-bf45-c15342026d24-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.126363 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.126334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78b56a34-ea76-4944-99b5-11b561194852-registry-certificates\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.127269 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.127234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78b56a34-ea76-4944-99b5-11b561194852-image-registry-private-configuration\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.127366 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.127241 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55fb2e48-ed73-44be-bf45-c15342026d24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.127452 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.127436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-registry-tls\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.127743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.127715 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78b56a34-ea76-4944-99b5-11b561194852-installation-pull-secrets\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.128163 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.128148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-serving-cert\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.133593 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.133545 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-427b6\" (UniqueName: \"kubernetes.io/projected/55fb2e48-ed73-44be-bf45-c15342026d24-kube-api-access-427b6\") pod \"cluster-monitoring-operator-5f7f494dd4-s5fns\" (UID: \"55fb2e48-ed73-44be-bf45-c15342026d24\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.133831 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.133812 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9966p\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-kube-api-access-9966p\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.134058 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.134037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78b56a34-ea76-4944-99b5-11b561194852-bound-sa-token\") pod \"image-registry-7d9c667bf6-bfgbb\" (UID: \"78b56a34-ea76-4944-99b5-11b561194852\") " pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.134133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.133709 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb7qn\" (UniqueName: \"kubernetes.io/projected/7c955eea-454f-483b-b3ab-4c96b2d4d8cd-kube-api-access-lb7qn\") pod \"insights-operator-68d9dd94cf-9xxq9\" (UID: \"7c955eea-454f-483b-b3ab-4c96b2d4d8cd\") " pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.134183 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.134134 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc6pl\" (UniqueName: \"kubernetes.io/projected/25ee3b41-af6d-4d49-a813-c9eee12fc550-kube-api-access-rc6pl\") pod \"downloads-68c9cbb497-w2vmq\" (UID: \"25ee3b41-af6d-4d49-a813-c9eee12fc550\") " pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:28.155817 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.155790 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" Jul 09 15:32:28.167618 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.167590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" Jul 09 15:32:28.171758 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.171732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:28.255538 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.255500 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:28.262957 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.262931 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" Jul 09 15:32:28.276039 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.275781 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:28.280275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.280077 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" Jul 09 15:32:28.310306 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.310186 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp"] Jul 09 15:32:28.362513 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.362375 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pj2r4" Jul 09 15:32:28.439857 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.439746 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" event={"ID":"17ce1faa-e7ec-48e1-9563-2af943642e22","Type":"ContainerStarted","Data":"f5727b7dec21179dc785ebf00fec3bcdbcdb06df3b7931078e6ee59bd4a6b075"} Jul 09 15:32:28.442412 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.441016 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-w2vmq"] Jul 09 15:32:28.448917 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:28.448884 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25ee3b41_af6d_4d49_a813_c9eee12fc550.slice/crio-58a13eb5a12fb8cd734fb08ff22335d7ccf68667a8f389c63be74068d3e78c50 WatchSource:0}: Error finding container 58a13eb5a12fb8cd734fb08ff22335d7ccf68667a8f389c63be74068d3e78c50: Status 404 returned error can't find the container with id 58a13eb5a12fb8cd734fb08ff22335d7ccf68667a8f389c63be74068d3e78c50 Jul 09 15:32:28.455925 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.455895 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns"] Jul 09 15:32:28.538916 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.538885 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm"] Jul 09 15:32:28.541619 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:28.541595 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea683fb5_2da7_4b3c_a8d9_42cfbfd03040.slice/crio-770047bc96f2bd88263b2ec140ca5eef2677ebc5af8fc452334c88c9071feae2 WatchSource:0}: Error finding container 770047bc96f2bd88263b2ec140ca5eef2677ebc5af8fc452334c88c9071feae2: Status 404 returned error can't find the container with id 770047bc96f2bd88263b2ec140ca5eef2677ebc5af8fc452334c88c9071feae2 Jul 09 15:32:28.546098 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.546077 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dc6bcc468-7nttr"] Jul 09 15:32:28.548563 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:28.548541 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08707e88_88a7_46f7_b9c7_80008e16b603.slice/crio-3a1e048eda160f9e473160b4ef88bc4e12c27782cffecf19961d8854b5498605 WatchSource:0}: Error finding container 3a1e048eda160f9e473160b4ef88bc4e12c27782cffecf19961d8854b5498605: Status 404 returned error can't find the container with id 3a1e048eda160f9e473160b4ef88bc4e12c27782cffecf19961d8854b5498605 Jul 09 15:32:28.672078 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.671999 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-9xxq9"] Jul 09 15:32:28.674478 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:28.674440 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d9c667bf6-bfgbb"] Jul 09 15:32:28.674877 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:28.674850 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c955eea_454f_483b_b3ab_4c96b2d4d8cd.slice/crio-a82a745c84a61ce8dfba1155aef7efb10bab6456fdba743db0dc6834ea86b4c4 WatchSource:0}: Error finding container a82a745c84a61ce8dfba1155aef7efb10bab6456fdba743db0dc6834ea86b4c4: Status 404 returned error can't find the container with id a82a745c84a61ce8dfba1155aef7efb10bab6456fdba743db0dc6834ea86b4c4 Jul 09 15:32:28.677162 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:28.677139 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b56a34_ea76_4944_99b5_11b561194852.slice/crio-9ff37f61bcf377130360282ac3ed7aa4f40b1d512e77a4bd970296d9bd0b311e WatchSource:0}: Error finding container 9ff37f61bcf377130360282ac3ed7aa4f40b1d512e77a4bd970296d9bd0b311e: Status 404 returned error can't find the container with id 9ff37f61bcf377130360282ac3ed7aa4f40b1d512e77a4bd970296d9bd0b311e Jul 09 15:32:29.447031 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.446812 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" event={"ID":"08707e88-88a7-46f7-b9c7-80008e16b603","Type":"ContainerStarted","Data":"e057b9fd74691c054038b5869e1b3cdaa5c52ca35429fb2a659fb473ea3e59ae"} Jul 09 15:32:29.447031 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.446910 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" event={"ID":"08707e88-88a7-46f7-b9c7-80008e16b603","Type":"ContainerStarted","Data":"3a1e048eda160f9e473160b4ef88bc4e12c27782cffecf19961d8854b5498605"} Jul 09 15:32:29.448926 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.448896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" event={"ID":"7c955eea-454f-483b-b3ab-4c96b2d4d8cd","Type":"ContainerStarted","Data":"a82a745c84a61ce8dfba1155aef7efb10bab6456fdba743db0dc6834ea86b4c4"} Jul 09 15:32:29.450473 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.450438 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" event={"ID":"55fb2e48-ed73-44be-bf45-c15342026d24","Type":"ContainerStarted","Data":"1eb4b6697826d0514a4f59a874dc2521b16287800bc42065c777a606834fd0c9"} Jul 09 15:32:29.451851 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.451803 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-w2vmq" event={"ID":"25ee3b41-af6d-4d49-a813-c9eee12fc550","Type":"ContainerStarted","Data":"58a13eb5a12fb8cd734fb08ff22335d7ccf68667a8f389c63be74068d3e78c50"} Jul 09 15:32:29.454255 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.454200 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" event={"ID":"ea683fb5-2da7-4b3c-a8d9-42cfbfd03040","Type":"ContainerStarted","Data":"770047bc96f2bd88263b2ec140ca5eef2677ebc5af8fc452334c88c9071feae2"} Jul 09 15:32:29.456229 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.456165 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" event={"ID":"78b56a34-ea76-4944-99b5-11b561194852","Type":"ContainerStarted","Data":"6921e78614e6ce13b474dcb91d304df66909c700c6f9938f3e23df7f4bc40a7a"} Jul 09 15:32:29.456229 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.456201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" event={"ID":"78b56a34-ea76-4944-99b5-11b561194852","Type":"ContainerStarted","Data":"9ff37f61bcf377130360282ac3ed7aa4f40b1d512e77a4bd970296d9bd0b311e"} Jul 09 15:32:29.456533 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.456496 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:29.471853 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.471014 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" podStartSLOduration=2.47099517 podStartE2EDuration="2.47099517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:29.470125447 +0000 UTC m=+66.007352623" watchObservedRunningTime="2026-07-09 15:32:29.47099517 +0000 UTC m=+66.008222344" Jul 09 15:32:29.494211 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:29.494131 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" podStartSLOduration=2.494114678 podStartE2EDuration="2.494114678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:29.492780827 +0000 UTC m=+66.030008000" watchObservedRunningTime="2026-07-09 15:32:29.494114678 +0000 UTC m=+66.031341873" Jul 09 15:32:30.172316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:30.172243 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:30.175362 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:30.175136 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:30.460038 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:30.459927 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:30.461499 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:30.461456 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6dc6bcc468-7nttr" Jul 09 15:32:31.423389 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:31.423369 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jvpzg" Jul 09 15:32:31.465897 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:31.465839 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" event={"ID":"7c955eea-454f-483b-b3ab-4c96b2d4d8cd","Type":"ContainerStarted","Data":"2d3dceee7d47ea22fac6e66fd716cd3c37176b342ca17aac0b1a588377f102f4"} Jul 09 15:32:31.468544 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:31.468223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" event={"ID":"ea683fb5-2da7-4b3c-a8d9-42cfbfd03040","Type":"ContainerStarted","Data":"af0b1280ea138c30b48e625c1654e5b67da9f6932a505c2ada2ff27804f4e65e"} Jul 09 15:32:31.470978 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:31.470886 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" event={"ID":"17ce1faa-e7ec-48e1-9563-2af943642e22","Type":"ContainerStarted","Data":"5ebe4956c3d1130c85bccbb4ce4feecead4e9cd7100d0550fe2b1aabf4ffcd5a"} Jul 09 15:32:31.483802 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:31.483695 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-9xxq9" podStartSLOduration=1.8231448590000001 podStartE2EDuration="4.483613792s" totalImagesPullingTime="2.660468933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:31.483214989 +0000 UTC m=+68.020442163" watchObservedRunningTime="2026-07-09 15:32:31.483613792 +0000 UTC m=+68.020840965" Jul 09 15:32:31.500732 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:31.500662 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nszjm" podStartSLOduration=1.711500781 podStartE2EDuration="4.500616381s" totalImagesPullingTime="2.7891156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:31.499345561 +0000 UTC m=+68.036572737" watchObservedRunningTime="2026-07-09 15:32:31.500616381 +0000 UTC m=+68.037843558" Jul 09 15:32:32.477191 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.477150 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" event={"ID":"17ce1faa-e7ec-48e1-9563-2af943642e22","Type":"ContainerStarted","Data":"da0fa73c825ca9135a7dd4769077fbeb2e8872cbb6c0dcd30d0f53c2231ca51a"} Jul 09 15:32:32.495481 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.495410 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-psbpp" podStartSLOduration=2.530781799 podStartE2EDuration="5.495390423s" totalImagesPullingTime="2.964608624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:32.494441441 +0000 UTC m=+69.031668615" watchObservedRunningTime="2026-07-09 15:32:32.495390423 +0000 UTC m=+69.032617597" Jul 09 15:32:32.724136 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.724102 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c869bbbbb-rt2wm"] Jul 09 15:32:32.743919 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.743832 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c869bbbbb-rt2wm"] Jul 09 15:32:32.744068 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.743943 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.746098 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.746055 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 15:32:32.746242 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.746129 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 15:32:32.746549 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.746323 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 15:32:32.746549 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.746494 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 15:32:32.746549 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.746535 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l8swt\"" Jul 09 15:32:32.746747 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.746456 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 15:32:32.769301 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.769273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:32:32.769449 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.769314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:32:32.771505 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.771482 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 15:32:32.771629 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.771486 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 15:32:32.782923 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.782897 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d-original-pull-secret\") pod \"global-pull-secret-syncer-8nz45\" (UID: \"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d\") " pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:32:32.783137 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.783116 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c3ed855-db93-4389-b437-eb12d222712b-metrics-certs\") pod \"network-metrics-daemon-s7k7t\" (UID: \"1c3ed855-db93-4389-b437-eb12d222712b\") " pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:32:32.846268 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.846224 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8nz45" Jul 09 15:32:32.852901 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.852878 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rsq6f\"" Jul 09 15:32:32.861538 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.861428 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7k7t" Jul 09 15:32:32.870163 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-oauth-config\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.870307 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:32.870307 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-oauth-serving-cert\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.870411 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-service-ca\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.870411 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-772hf\" (UniqueName: \"kubernetes.io/projected/62f47351-9497-41df-9f3c-4afccd76f9e1-kube-api-access-772hf\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.870411 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-console-config\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.870411 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.870399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-serving-cert\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.873078 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.872993 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 15:32:32.882317 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.882287 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 15:32:32.893760 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.893731 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvpvz\" (UniqueName: \"kubernetes.io/projected/207bf408-a81a-47e7-8f81-bf939ad0227a-kube-api-access-dvpvz\") pod \"network-check-target-wx6lp\" (UID: \"207bf408-a81a-47e7-8f81-bf939ad0227a\") " pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.971369 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-oauth-serving-cert\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.971430 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-service-ca\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.971456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-772hf\" (UniqueName: \"kubernetes.io/projected/62f47351-9497-41df-9f3c-4afccd76f9e1-kube-api-access-772hf\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.972973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-console-config\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.973028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-serving-cert\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.973076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-oauth-config\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973360 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.973280 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-service-ca\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.973958 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.973929 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-console-config\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.974103 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.974080 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-oauth-serving-cert\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.981500 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.978991 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-oauth-config\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.983716 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.983424 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mw449"] Jul 09 15:32:32.988101 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.985335 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-serving-cert\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:32.998489 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:32.998367 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-772hf\" (UniqueName: \"kubernetes.io/projected/62f47351-9497-41df-9f3c-4afccd76f9e1-kube-api-access-772hf\") pod \"console-6c869bbbbb-rt2wm\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:33.002216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.000497 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mw449"] Jul 09 15:32:33.002216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.000627 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.007490 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.007007 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f9q7c\"" Jul 09 15:32:33.007490 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.007278 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 15:32:33.007965 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.007770 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 15:32:33.030392 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.030324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8nz45"] Jul 09 15:32:33.033336 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:33.033307 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b7bb8a_a0ca_4433_aa81_bf03c30c5a6d.slice/crio-e83eac9ed6900296f6fc9c9a25d0ddf2f7ab9f4532768381d4acf3a69e5357c1 WatchSource:0}: Error finding container e83eac9ed6900296f6fc9c9a25d0ddf2f7ab9f4532768381d4acf3a69e5357c1: Status 404 returned error can't find the container with id e83eac9ed6900296f6fc9c9a25d0ddf2f7ab9f4532768381d4acf3a69e5357c1 Jul 09 15:32:33.053944 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.053911 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7k7t"] Jul 09 15:32:33.055395 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.055317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:33.057517 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:33.057488 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3ed855_db93_4389_b437_eb12d222712b.slice/crio-207f838be98f653fc2511644085379a74df8604fb716b5ebd94929f944585ec3 WatchSource:0}: Error finding container 207f838be98f653fc2511644085379a74df8604fb716b5ebd94929f944585ec3: Status 404 returned error can't find the container with id 207f838be98f653fc2511644085379a74df8604fb716b5ebd94929f944585ec3 Jul 09 15:32:33.074147 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.074081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbwbg\" (UniqueName: \"kubernetes.io/projected/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-api-access-lbwbg\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.074147 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.074133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-data-volume\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.074388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.074206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.074388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.074262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-crio-socket\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.074388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.074350 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.141929 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.141889 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnkz7\"" Jul 09 15:32:33.150218 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.150188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:33.175293 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.175258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.175508 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.175331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbwbg\" (UniqueName: \"kubernetes.io/projected/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-api-access-lbwbg\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.175508 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.175375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-data-volume\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.175508 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.175402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.175508 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.175427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-crio-socket\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.175508 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.175480 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 15:32:33.175762 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.175587 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:33.675536619 +0000 UTC m=+70.212763785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : secret "insights-runtime-extractor-tls" not found Jul 09 15:32:33.175762 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.175583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-crio-socket\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.175895 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.175785 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 15:32:33.175895 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.175838 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:33.675823479 +0000 UTC m=+70.213050635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : configmap "kube-rbac-proxy" not found Jul 09 15:32:33.176141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.176108 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-data-volume\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.186911 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.186866 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbwbg\" (UniqueName: \"kubernetes.io/projected/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-api-access-lbwbg\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.205636 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.205604 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c869bbbbb-rt2wm"] Jul 09 15:32:33.209653 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:33.209619 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f47351_9497_41df_9f3c_4afccd76f9e1.slice/crio-9ee4dd3f8066e7176dfd18c0214a9b2dcbe912393950bd74336e785d9313dac3 WatchSource:0}: Error finding container 9ee4dd3f8066e7176dfd18c0214a9b2dcbe912393950bd74336e785d9313dac3: Status 404 returned error can't find the container with id 9ee4dd3f8066e7176dfd18c0214a9b2dcbe912393950bd74336e785d9313dac3 Jul 09 15:32:33.299794 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.299699 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wx6lp"] Jul 09 15:32:33.303425 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:33.303398 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207bf408_a81a_47e7_8f81_bf939ad0227a.slice/crio-54e28ef6ff92cef6e860f72df8e5fffa77a546fb77ed137321065c83712de034 WatchSource:0}: Error finding container 54e28ef6ff92cef6e860f72df8e5fffa77a546fb77ed137321065c83712de034: Status 404 returned error can't find the container with id 54e28ef6ff92cef6e860f72df8e5fffa77a546fb77ed137321065c83712de034 Jul 09 15:32:33.481968 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.481929 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wx6lp" event={"ID":"207bf408-a81a-47e7-8f81-bf939ad0227a","Type":"ContainerStarted","Data":"54e28ef6ff92cef6e860f72df8e5fffa77a546fb77ed137321065c83712de034"} Jul 09 15:32:33.483367 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.483339 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7k7t" event={"ID":"1c3ed855-db93-4389-b437-eb12d222712b","Type":"ContainerStarted","Data":"207f838be98f653fc2511644085379a74df8604fb716b5ebd94929f944585ec3"} Jul 09 15:32:33.484724 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.484699 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c869bbbbb-rt2wm" event={"ID":"62f47351-9497-41df-9f3c-4afccd76f9e1","Type":"ContainerStarted","Data":"9ee4dd3f8066e7176dfd18c0214a9b2dcbe912393950bd74336e785d9313dac3"} Jul 09 15:32:33.485910 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.485878 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8nz45" event={"ID":"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d","Type":"ContainerStarted","Data":"e83eac9ed6900296f6fc9c9a25d0ddf2f7ab9f4532768381d4acf3a69e5357c1"} Jul 09 15:32:33.679698 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.679531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.679698 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:33.679622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:33.679698 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.679680 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 15:32:33.680020 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.679712 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 15:32:33.680020 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.679749 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:34.679731125 +0000 UTC m=+71.216958281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : secret "insights-runtime-extractor-tls" not found Jul 09 15:32:33.680020 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:33.679768 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:34.679758137 +0000 UTC m=+71.216985295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : configmap "kube-rbac-proxy" not found Jul 09 15:32:34.688308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:34.688272 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:34.688755 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:34.688357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:34.688755 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:34.688482 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 15:32:34.688755 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:34.688553 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:36.688530296 +0000 UTC m=+73.225757462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : configmap "kube-rbac-proxy" not found Jul 09 15:32:34.688755 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:34.688651 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 15:32:34.688755 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:34.688685 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:36.68867485 +0000 UTC m=+73.225902002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : secret "insights-runtime-extractor-tls" not found Jul 09 15:32:35.506801 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:35.506760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7k7t" event={"ID":"1c3ed855-db93-4389-b437-eb12d222712b","Type":"ContainerStarted","Data":"67a48f77f9e4a6b34d162a8501e378cbf96e634c0a89049e857ce72feb54348a"} Jul 09 15:32:35.506985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:35.506865 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7k7t" event={"ID":"1c3ed855-db93-4389-b437-eb12d222712b","Type":"ContainerStarted","Data":"30cad6bd76cc6f95436ea31a26c1921245d13e22e4befef1740374318b0b1253"} Jul 09 15:32:35.539857 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:35.539107 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s7k7t" podStartSLOduration=66.944743105 podStartE2EDuration="1m8.539083882s" totalImagesPullingTime="1.594340777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:35.537088035 +0000 UTC m=+72.074315211" watchObservedRunningTime="2026-07-09 15:32:35.539083882 +0000 UTC m=+72.076311059" Jul 09 15:32:36.587003 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:36.586972 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvpzg_f46785e3-38b5-4c15-8891-270e1983b38e/dns/0.log" Jul 09 15:32:36.711628 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:36.711593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:36.711843 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:36.711664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:36.711843 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:36.711735 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 15:32:36.711843 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:36.711765 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 15:32:36.711843 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:36.711818 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:40.711798319 +0000 UTC m=+77.249025477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : secret "insights-runtime-extractor-tls" not found Jul 09 15:32:36.711843 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:36.711835 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:40.711826681 +0000 UTC m=+77.249053839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : configmap "kube-rbac-proxy" not found Jul 09 15:32:36.787042 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:36.787012 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvpzg_f46785e3-38b5-4c15-8891-270e1983b38e/kube-rbac-proxy/0.log" Jul 09 15:32:36.988169 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:36.988124 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4zmmk_45f9fd98-9365-48d7-a594-1ca287fd2234/dns-node-resolver/0.log" Jul 09 15:32:37.986792 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:37.986758 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d9c667bf6-bfgbb_78b56a34-ea76-4944-99b5-11b561194852/registry/0.log" Jul 09 15:32:38.185118 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:38.185022 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cz222_e873854a-76a3-4172-ae5e-bbd3af1a04b7/node-ca/0.log" Jul 09 15:32:38.786965 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:38.786887 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dc6bcc468-7nttr_08707e88-88a7-46f7-b9c7-80008e16b603/router/0.log" Jul 09 15:32:38.989035 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:38.989004 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5lgqr_200101ee-c3b4-49b8-93da-a6cf99fa6061/serve-healthcheck-canary/0.log" Jul 09 15:32:40.532099 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.532065 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c869bbbbb-rt2wm" event={"ID":"62f47351-9497-41df-9f3c-4afccd76f9e1","Type":"ContainerStarted","Data":"14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849"} Jul 09 15:32:40.533674 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.533647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8nz45" event={"ID":"b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d","Type":"ContainerStarted","Data":"3baacf0af3ce448f58b098783f4c1e40f17adfbcae2a1370de10a40df56c382a"} Jul 09 15:32:40.535092 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.535030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wx6lp" event={"ID":"207bf408-a81a-47e7-8f81-bf939ad0227a","Type":"ContainerStarted","Data":"12f01734ce7fa9fc459a84c94920b0beb0f2ff1c27a49d8825c1a22d1e39a48e"} Jul 09 15:32:40.535188 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.535132 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:32:40.553745 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.553687 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c869bbbbb-rt2wm" podStartSLOduration=1.466473595 podStartE2EDuration="8.553665495s" totalImagesPullingTime="7.0871919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:40.552510898 +0000 UTC m=+77.089738068" watchObservedRunningTime="2026-07-09 15:32:40.553665495 +0000 UTC m=+77.090892753" Jul 09 15:32:40.575648 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.575586 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wx6lp" podStartSLOduration=66.573255491 podStartE2EDuration="1m13.57556524s" totalImagesPullingTime="7.002309749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:40.575088641 +0000 UTC m=+77.112315817" watchObservedRunningTime="2026-07-09 15:32:40.57556524 +0000 UTC m=+77.112792415" Jul 09 15:32:40.594832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.594778 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8nz45" podStartSLOduration=66.317514959 podStartE2EDuration="1m13.594758253s" totalImagesPullingTime="7.277243294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:40.594759678 +0000 UTC m=+77.131986853" watchObservedRunningTime="2026-07-09 15:32:40.594758253 +0000 UTC m=+77.131985426" Jul 09 15:32:40.747053 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.747013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:40.747254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:40.747100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:40.747254 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:40.747108 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 15:32:40.747254 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:40.747183 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:48.747162175 +0000 UTC m=+85.284389333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : configmap "kube-rbac-proxy" not found Jul 09 15:32:40.747389 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:40.747262 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 15:32:40.747389 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:40.747330 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls podName:2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8 nodeName:}" failed. No retries permitted until 2026-07-09 15:32:48.74731194 +0000 UTC m=+85.284539105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mw449" (UID: "2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8") : secret "insights-runtime-extractor-tls" not found Jul 09 15:32:43.055799 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:43.055758 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:43.056305 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:43.055831 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:43.061973 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:43.061937 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:43.552554 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:43.552521 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:32:46.266144 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:46.266010 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c869bbbbb-rt2wm"] Jul 09 15:32:48.820536 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:48.820499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:48.821016 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:48.820602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:48.821187 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:48.821167 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:48.823122 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:48.823097 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mw449\" (UID: \"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8\") " pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:48.926611 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:48.926568 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mw449" Jul 09 15:32:49.945168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:49.945134 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mw449"] Jul 09 15:32:49.948508 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:49.948454 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8ee5ad_e3a6_4b98_a568_fdcf98b0c4f8.slice/crio-4acc9e333a14aab526a99cbb3e41ccd8517ccdfe077ccbc9c803091d69921058 WatchSource:0}: Error finding container 4acc9e333a14aab526a99cbb3e41ccd8517ccdfe077ccbc9c803091d69921058: Status 404 returned error can't find the container with id 4acc9e333a14aab526a99cbb3e41ccd8517ccdfe077ccbc9c803091d69921058 Jul 09 15:32:50.390420 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.390379 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c"] Jul 09 15:32:50.400637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.399225 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:50.402602 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.402576 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 15:32:50.402936 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.402913 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-chz7p\"" Jul 09 15:32:50.405804 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.405779 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c"] Jul 09 15:32:50.465795 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.465765 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d9c667bf6-bfgbb" Jul 09 15:32:50.533877 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.533840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c381064-16b8-45e0-b61e-b427bc74085f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-jqk9c\" (UID: \"5c381064-16b8-45e0-b61e-b427bc74085f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:50.577351 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.577310 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" event={"ID":"55fb2e48-ed73-44be-bf45-c15342026d24","Type":"ContainerStarted","Data":"df1bde64d060f944a9fb35601a5a8611bc004bdf08d6f4094e03f207aafcfd48"} Jul 09 15:32:50.580431 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.580397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-w2vmq" event={"ID":"25ee3b41-af6d-4d49-a813-c9eee12fc550","Type":"ContainerStarted","Data":"623ba7b70a31986e31239e064a68b0e3e96cd89e1adaf308590e21b1430dfc09"} Jul 09 15:32:50.580668 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.580642 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:50.583301 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.583080 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mw449" event={"ID":"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8","Type":"ContainerStarted","Data":"4acc9e333a14aab526a99cbb3e41ccd8517ccdfe077ccbc9c803091d69921058"} Jul 09 15:32:50.592408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.592367 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-w2vmq" Jul 09 15:32:50.594475 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.594405 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-s5fns" podStartSLOduration=2.24745818 podStartE2EDuration="23.594388871s" totalImagesPullingTime="21.346930691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:50.59272569 +0000 UTC m=+87.129952879" watchObservedRunningTime="2026-07-09 15:32:50.594388871 +0000 UTC m=+87.131616044" Jul 09 15:32:50.608302 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.608242 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-w2vmq" podStartSLOduration=2.195628832 podStartE2EDuration="23.608223607s" totalImagesPullingTime="21.412594775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:50.607502571 +0000 UTC m=+87.144729746" watchObservedRunningTime="2026-07-09 15:32:50.608223607 +0000 UTC m=+87.145450782" Jul 09 15:32:50.636962 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:50.635235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c381064-16b8-45e0-b61e-b427bc74085f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-jqk9c\" (UID: \"5c381064-16b8-45e0-b61e-b427bc74085f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:50.636962 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:50.636396 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 09 15:32:50.636962 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:32:50.636494 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c381064-16b8-45e0-b61e-b427bc74085f-tls-certificates podName:5c381064-16b8-45e0-b61e-b427bc74085f nodeName:}" failed. No retries permitted until 2026-07-09 15:32:51.136452876 +0000 UTC m=+87.673680044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5c381064-16b8-45e0-b61e-b427bc74085f-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-jqk9c" (UID: "5c381064-16b8-45e0-b61e-b427bc74085f") : secret "prometheus-operator-admission-webhook-tls" not found Jul 09 15:32:51.139280 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:51.139237 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c381064-16b8-45e0-b61e-b427bc74085f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-jqk9c\" (UID: \"5c381064-16b8-45e0-b61e-b427bc74085f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:51.142256 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:51.142220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c381064-16b8-45e0-b61e-b427bc74085f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-jqk9c\" (UID: \"5c381064-16b8-45e0-b61e-b427bc74085f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:51.317302 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:51.317264 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:51.497268 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:51.496977 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c"] Jul 09 15:32:51.500948 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:32:51.500907 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c381064_16b8_45e0_b61e_b427bc74085f.slice/crio-2c0eb14b3d4c153618ff25f83bede6ed616fe8b4a72b30d065c4592e559bc4a9 WatchSource:0}: Error finding container 2c0eb14b3d4c153618ff25f83bede6ed616fe8b4a72b30d065c4592e559bc4a9: Status 404 returned error can't find the container with id 2c0eb14b3d4c153618ff25f83bede6ed616fe8b4a72b30d065c4592e559bc4a9 Jul 09 15:32:51.590369 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:51.590324 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" event={"ID":"5c381064-16b8-45e0-b61e-b427bc74085f","Type":"ContainerStarted","Data":"2c0eb14b3d4c153618ff25f83bede6ed616fe8b4a72b30d065c4592e559bc4a9"} Jul 09 15:32:57.617624 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:57.617580 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mw449" event={"ID":"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8","Type":"ContainerStarted","Data":"2272b905432db20c9990c7648110c609b10dfbed995e69e9b73448b579748e33"} Jul 09 15:32:58.623304 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:58.623264 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" event={"ID":"5c381064-16b8-45e0-b61e-b427bc74085f","Type":"ContainerStarted","Data":"de24f86c8c518cefc7fcbf1bfb76e130ee31e49e6c620e9827a889310fc06190"} Jul 09 15:32:58.623775 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:58.623497 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:58.628980 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:58.628954 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" Jul 09 15:32:58.640293 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:58.640249 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-jqk9c" podStartSLOduration=2.509463297 podStartE2EDuration="8.640234171s" totalImagesPullingTime="6.130770874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:58.639455325 +0000 UTC m=+95.176682503" watchObservedRunningTime="2026-07-09 15:32:58.640234171 +0000 UTC m=+95.177461344" Jul 09 15:32:59.629091 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:32:59.629058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mw449" event={"ID":"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8","Type":"ContainerStarted","Data":"9344124ce22a258394fbcf8ecaa0a95bdeb0a842751220ab848222eccd2418ec"} Jul 09 15:33:03.981557 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:03.981190 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wmvr9"] Jul 09 15:33:04.005863 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.005332 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m5m6f"] Jul 09 15:33:04.005863 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.005447 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.008398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.008376 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 15:33:04.008640 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.008611 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4hlld\"" Jul 09 15:33:04.008997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.008979 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 15:33:04.009425 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.009281 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 15:33:04.009425 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.009293 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 15:33:04.025349 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.025327 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wmvr9"] Jul 09 15:33:04.025499 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.025478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.035656 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.035634 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tffth\"" Jul 09 15:33:04.035940 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.035923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 15:33:04.036075 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.036057 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 15:33:04.036354 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.036339 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 15:33:04.055506 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-sys\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055634 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055524 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055634 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-textfile\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055634 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-wtmp\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055633 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-root\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055686 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/12199445-7494-49e4-aa80-3f6f435007dd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.055798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m5mw\" (UniqueName: \"kubernetes.io/projected/e71e8e47-9ac8-4a5b-af1e-f1182509f332-kube-api-access-5m5mw\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7vpc\" (UniqueName: \"kubernetes.io/projected/12199445-7494-49e4-aa80-3f6f435007dd-kube-api-access-v7vpc\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.055798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12199445-7494-49e4-aa80-3f6f435007dd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.055974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-tls\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055890 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e71e8e47-9ac8-4a5b-af1e-f1182509f332-metrics-client-ca\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.055974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.055974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.056132 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.055975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157092 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-wtmp\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157260 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-root\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157260 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/12199445-7494-49e4-aa80-3f6f435007dd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157260 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-root\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157421 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m5mw\" (UniqueName: \"kubernetes.io/projected/e71e8e47-9ac8-4a5b-af1e-f1182509f332-kube-api-access-5m5mw\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157421 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7vpc\" (UniqueName: \"kubernetes.io/projected/12199445-7494-49e4-aa80-3f6f435007dd-kube-api-access-v7vpc\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157421 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12199445-7494-49e4-aa80-3f6f435007dd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157421 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-tls\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157421 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157399 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e71e8e47-9ac8-4a5b-af1e-f1182509f332-metrics-client-ca\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/12199445-7494-49e4-aa80-3f6f435007dd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157630 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-sys\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.157690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.158020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.157726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-textfile\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.158020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.158002 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-textfile\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.158133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.158084 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-wtmp\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.158562 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.158504 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.158669 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.158581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e71e8e47-9ac8-4a5b-af1e-f1182509f332-sys\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.158669 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.158603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.158669 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:33:04.158658 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 15:33:04.158825 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:33:04.158715 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-tls podName:12199445-7494-49e4-aa80-3f6f435007dd nodeName:}" failed. No retries permitted until 2026-07-09 15:33:04.658696781 +0000 UTC m=+101.195923937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-wmvr9" (UID: "12199445-7494-49e4-aa80-3f6f435007dd") : secret "kube-state-metrics-tls" not found Jul 09 15:33:04.159076 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.159053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12199445-7494-49e4-aa80-3f6f435007dd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.159180 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.159159 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e71e8e47-9ac8-4a5b-af1e-f1182509f332-metrics-client-ca\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.161205 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.161067 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.161299 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.161281 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-tls\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.161634 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.161612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e71e8e47-9ac8-4a5b-af1e-f1182509f332-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.169569 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.169524 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m5mw\" (UniqueName: \"kubernetes.io/projected/e71e8e47-9ac8-4a5b-af1e-f1182509f332-kube-api-access-5m5mw\") pod \"node-exporter-m5m6f\" (UID: \"e71e8e47-9ac8-4a5b-af1e-f1182509f332\") " pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.170755 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.170735 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7vpc\" (UniqueName: \"kubernetes.io/projected/12199445-7494-49e4-aa80-3f6f435007dd-kube-api-access-v7vpc\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.337503 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.337408 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m5m6f" Jul 09 15:33:04.661978 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.661935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.664771 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.664742 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/12199445-7494-49e4-aa80-3f6f435007dd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wmvr9\" (UID: \"12199445-7494-49e4-aa80-3f6f435007dd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:04.918806 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:04.918679 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" Jul 09 15:33:05.365071 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:33:05.365041 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71e8e47_9ac8_4a5b_af1e_f1182509f332.slice/crio-26ad1a22a3987b0630cf192d5069eee55af3fee26dc248fbe5f46118571a6345 WatchSource:0}: Error finding container 26ad1a22a3987b0630cf192d5069eee55af3fee26dc248fbe5f46118571a6345: Status 404 returned error can't find the container with id 26ad1a22a3987b0630cf192d5069eee55af3fee26dc248fbe5f46118571a6345 Jul 09 15:33:05.499055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:05.499031 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wmvr9"] Jul 09 15:33:05.501111 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:33:05.501084 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12199445_7494_49e4_aa80_3f6f435007dd.slice/crio-ba824ccba25eb189f88aae4bc6863d388a00476f80e6624b8198c8da481eef75 WatchSource:0}: Error finding container ba824ccba25eb189f88aae4bc6863d388a00476f80e6624b8198c8da481eef75: Status 404 returned error can't find the container with id ba824ccba25eb189f88aae4bc6863d388a00476f80e6624b8198c8da481eef75 Jul 09 15:33:05.654254 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:05.654221 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" event={"ID":"12199445-7494-49e4-aa80-3f6f435007dd","Type":"ContainerStarted","Data":"ba824ccba25eb189f88aae4bc6863d388a00476f80e6624b8198c8da481eef75"} Jul 09 15:33:05.655359 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:05.655334 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5m6f" event={"ID":"e71e8e47-9ac8-4a5b-af1e-f1182509f332","Type":"ContainerStarted","Data":"26ad1a22a3987b0630cf192d5069eee55af3fee26dc248fbe5f46118571a6345"} Jul 09 15:33:06.661774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:06.661730 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mw449" event={"ID":"2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8","Type":"ContainerStarted","Data":"c89b1a25e976ca003b70314a0777946437be928122a8bf89fea1215984c24e2f"} Jul 09 15:33:06.682936 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:06.682841 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mw449" podStartSLOduration=19.23211115 podStartE2EDuration="34.682820474s" totalImagesPullingTime="15.450709324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:32:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:06.682039601 +0000 UTC m=+103.219266775" watchObservedRunningTime="2026-07-09 15:33:06.682820474 +0000 UTC m=+103.220047650" Jul 09 15:33:07.072082 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.072048 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l"] Jul 09 15:33:07.090479 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.090431 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l"] Jul 09 15:33:07.090630 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.090607 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.093186 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.092963 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-151r8ibl9tis7\"" Jul 09 15:33:07.093186 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.093036 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 15:33:07.093186 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.093086 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 15:33:07.093418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.093289 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 15:33:07.093418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.093328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sqjdf\"" Jul 09 15:33:07.094141 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.093785 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 15:33:07.094661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.094450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 15:33:07.182697 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.182697 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-grpc-tls\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.183033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.183033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.183033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-tls\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.183033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.183033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59ffaac6-e6df-43e4-b257-41d7f26c4312-metrics-client-ca\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.183033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.182884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm87s\" (UniqueName: \"kubernetes.io/projected/59ffaac6-e6df-43e4-b257-41d7f26c4312-kube-api-access-lm87s\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.283960 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.283920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.283972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-grpc-tls\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-tls\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284142 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59ffaac6-e6df-43e4-b257-41d7f26c4312-metrics-client-ca\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.284496 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284157 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm87s\" (UniqueName: \"kubernetes.io/projected/59ffaac6-e6df-43e4-b257-41d7f26c4312-kube-api-access-lm87s\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.285008 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.284954 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59ffaac6-e6df-43e4-b257-41d7f26c4312-metrics-client-ca\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.286955 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.286931 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.287080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.287054 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.287132 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.287085 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.287407 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.287384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.287451 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.287391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-thanos-querier-tls\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.287991 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.287974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59ffaac6-e6df-43e4-b257-41d7f26c4312-secret-grpc-tls\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.295292 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.295270 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm87s\" (UniqueName: \"kubernetes.io/projected/59ffaac6-e6df-43e4-b257-41d7f26c4312-kube-api-access-lm87s\") pod \"thanos-querier-5bbd9b4c9-gfn4l\" (UID: \"59ffaac6-e6df-43e4-b257-41d7f26c4312\") " pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.402752 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.402728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:07.578347 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.578321 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l"] Jul 09 15:33:07.585656 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:33:07.585623 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ffaac6_e6df_43e4_b257_41d7f26c4312.slice/crio-3ef49bfd008e20cb92dc8c8574063af2c904da4bfbfce0ce5eb6c01d9bb22bca WatchSource:0}: Error finding container 3ef49bfd008e20cb92dc8c8574063af2c904da4bfbfce0ce5eb6c01d9bb22bca: Status 404 returned error can't find the container with id 3ef49bfd008e20cb92dc8c8574063af2c904da4bfbfce0ce5eb6c01d9bb22bca Jul 09 15:33:07.667267 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.667239 2577 generic.go:356] "Generic (PLEG): container finished" podID="e71e8e47-9ac8-4a5b-af1e-f1182509f332" containerID="11011a2b6de53b4532e731905852536541b810e31dfce8340d555b183d096f35" exitCode=0 Jul 09 15:33:07.667679 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.667321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5m6f" event={"ID":"e71e8e47-9ac8-4a5b-af1e-f1182509f332","Type":"ContainerDied","Data":"11011a2b6de53b4532e731905852536541b810e31dfce8340d555b183d096f35"} Jul 09 15:33:07.668748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:07.668727 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"3ef49bfd008e20cb92dc8c8574063af2c904da4bfbfce0ce5eb6c01d9bb22bca"} Jul 09 15:33:08.664400 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.664364 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f7896b567-5fjq8"] Jul 09 15:33:08.684791 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.684759 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" event={"ID":"12199445-7494-49e4-aa80-3f6f435007dd","Type":"ContainerStarted","Data":"217bd1a1d3657667278e53c19c52ea09307050bda65080ea993201b2943eec8b"} Jul 09 15:33:08.685124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.684804 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7896b567-5fjq8"] Jul 09 15:33:08.685124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.684893 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.685124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.684901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" event={"ID":"12199445-7494-49e4-aa80-3f6f435007dd","Type":"ContainerStarted","Data":"c35473424bd83dac6d5936167098c17327148f30e877c4270b6d14030120e65c"} Jul 09 15:33:08.685124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.684976 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" event={"ID":"12199445-7494-49e4-aa80-3f6f435007dd","Type":"ContainerStarted","Data":"6dd5289c81f33295e3ae02a836fc59b8d8de8d98dd1f6fd055aec2fb97be40fe"} Jul 09 15:33:08.685124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.684996 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5m6f" event={"ID":"e71e8e47-9ac8-4a5b-af1e-f1182509f332","Type":"ContainerStarted","Data":"bdbd83685ecfc176f783fedc1940edb16bbf3cc5fad3f15f68021c7cf29113ec"} Jul 09 15:33:08.685124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.685013 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5m6f" event={"ID":"e71e8e47-9ac8-4a5b-af1e-f1182509f332","Type":"ContainerStarted","Data":"023ac0de0faa736b04a841216631ff92f4a1d021c19aead5acd1e06db4fd457a"} Jul 09 15:33:08.693085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.693059 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 15:33:08.696279 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-oauth-serving-cert\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.696378 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-trusted-ca-bundle\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.696424 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-serving-cert\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.696424 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxrrr\" (UniqueName: \"kubernetes.io/projected/536ea1cf-3191-4713-8a32-e341388008cf-kube-api-access-zxrrr\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.696967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-oauth-config\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.696967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-service-ca\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.696967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.696957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-console-config\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.698633 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.698596 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-wmvr9" podStartSLOduration=3.605863764 podStartE2EDuration="5.698582801s" totalImagesPullingTime="2.092719037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:08.696399868 +0000 UTC m=+105.233627056" watchObservedRunningTime="2026-07-09 15:33:08.698582801 +0000 UTC m=+105.235809974" Jul 09 15:33:08.758458 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.758159 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m5m6f" podStartSLOduration=4.143962296 podStartE2EDuration="5.758140279s" totalImagesPullingTime="1.527137361s" totalInitContainerRuntime="87.040622ms" isStatefulPod=true podCreationTimestamp="2026-07-09 15:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:08.716557567 +0000 UTC m=+105.253784732" watchObservedRunningTime="2026-07-09 15:33:08.758140279 +0000 UTC m=+105.295367456" Jul 09 15:33:08.798227 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-oauth-config\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.798227 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-service-ca\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.798498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-console-config\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.798498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-oauth-serving-cert\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.798498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798320 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-trusted-ca-bundle\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.798498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-serving-cert\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.798498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.798385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxrrr\" (UniqueName: \"kubernetes.io/projected/536ea1cf-3191-4713-8a32-e341388008cf-kube-api-access-zxrrr\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.799126 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.799096 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-oauth-serving-cert\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.799232 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.799107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-console-config\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.799301 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.799239 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-trusted-ca-bundle\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.799387 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.799367 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-service-ca\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.800866 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.800844 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-oauth-config\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.800939 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.800900 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-serving-cert\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.810606 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.810583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxrrr\" (UniqueName: \"kubernetes.io/projected/536ea1cf-3191-4713-8a32-e341388008cf-kube-api-access-zxrrr\") pod \"console-5f7896b567-5fjq8\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:08.995042 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:08.994957 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:09.154108 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:09.154030 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7896b567-5fjq8"] Jul 09 15:33:09.156163 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:33:09.156137 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536ea1cf_3191_4713_8a32_e341388008cf.slice/crio-3075ea8befb9ada7b0a0177dc8ee8f22f10db42d4307c567da2bd29172183f42 WatchSource:0}: Error finding container 3075ea8befb9ada7b0a0177dc8ee8f22f10db42d4307c567da2bd29172183f42: Status 404 returned error can't find the container with id 3075ea8befb9ada7b0a0177dc8ee8f22f10db42d4307c567da2bd29172183f42 Jul 09 15:33:09.683395 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:09.683359 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7896b567-5fjq8" event={"ID":"536ea1cf-3191-4713-8a32-e341388008cf","Type":"ContainerStarted","Data":"855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f"} Jul 09 15:33:09.683578 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:09.683418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7896b567-5fjq8" event={"ID":"536ea1cf-3191-4713-8a32-e341388008cf","Type":"ContainerStarted","Data":"3075ea8befb9ada7b0a0177dc8ee8f22f10db42d4307c567da2bd29172183f42"} Jul 09 15:33:09.704313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:09.704260 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f7896b567-5fjq8" podStartSLOduration=1.704243515 podStartE2EDuration="1.704243515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:09.70300088 +0000 UTC m=+106.240228052" watchObservedRunningTime="2026-07-09 15:33:09.704243515 +0000 UTC m=+106.241470688" Jul 09 15:33:11.290166 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.290123 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c869bbbbb-rt2wm" podUID="62f47351-9497-41df-9f3c-4afccd76f9e1" containerName="console" containerID="cri-o://14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849" gracePeriod=15 Jul 09 15:33:11.542316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.542231 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wx6lp" Jul 09 15:33:11.591558 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.591519 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:33:11.622119 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622091 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-serving-cert\") pod \"62f47351-9497-41df-9f3c-4afccd76f9e1\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " Jul 09 15:33:11.622289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622132 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-oauth-config\") pod \"62f47351-9497-41df-9f3c-4afccd76f9e1\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " Jul 09 15:33:11.622289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622163 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-772hf\" (UniqueName: \"kubernetes.io/projected/62f47351-9497-41df-9f3c-4afccd76f9e1-kube-api-access-772hf\") pod \"62f47351-9497-41df-9f3c-4afccd76f9e1\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " Jul 09 15:33:11.622289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622195 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-console-config\") pod \"62f47351-9497-41df-9f3c-4afccd76f9e1\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " Jul 09 15:33:11.622289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622219 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-oauth-serving-cert\") pod \"62f47351-9497-41df-9f3c-4afccd76f9e1\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " Jul 09 15:33:11.622289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622251 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-service-ca\") pod \"62f47351-9497-41df-9f3c-4afccd76f9e1\" (UID: \"62f47351-9497-41df-9f3c-4afccd76f9e1\") " Jul 09 15:33:11.622842 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622809 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-console-config" (OuterVolumeSpecName: "console-config") pod "62f47351-9497-41df-9f3c-4afccd76f9e1" (UID: "62f47351-9497-41df-9f3c-4afccd76f9e1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:33:11.622842 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622835 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62f47351-9497-41df-9f3c-4afccd76f9e1" (UID: "62f47351-9497-41df-9f3c-4afccd76f9e1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:33:11.623024 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.622876 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-service-ca" (OuterVolumeSpecName: "service-ca") pod "62f47351-9497-41df-9f3c-4afccd76f9e1" (UID: "62f47351-9497-41df-9f3c-4afccd76f9e1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:33:11.624779 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.624750 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62f47351-9497-41df-9f3c-4afccd76f9e1" (UID: "62f47351-9497-41df-9f3c-4afccd76f9e1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:33:11.624904 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.624784 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f47351-9497-41df-9f3c-4afccd76f9e1-kube-api-access-772hf" (OuterVolumeSpecName: "kube-api-access-772hf") pod "62f47351-9497-41df-9f3c-4afccd76f9e1" (UID: "62f47351-9497-41df-9f3c-4afccd76f9e1"). InnerVolumeSpecName "kube-api-access-772hf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:33:11.624904 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.624794 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62f47351-9497-41df-9f3c-4afccd76f9e1" (UID: "62f47351-9497-41df-9f3c-4afccd76f9e1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:33:11.692890 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.692853 2577 generic.go:356] "Generic (PLEG): container finished" podID="62f47351-9497-41df-9f3c-4afccd76f9e1" containerID="14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849" exitCode=0 Jul 09 15:33:11.693083 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.692911 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c869bbbbb-rt2wm" Jul 09 15:33:11.693083 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.692923 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c869bbbbb-rt2wm" event={"ID":"62f47351-9497-41df-9f3c-4afccd76f9e1","Type":"ContainerDied","Data":"14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849"} Jul 09 15:33:11.693083 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.692990 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c869bbbbb-rt2wm" event={"ID":"62f47351-9497-41df-9f3c-4afccd76f9e1","Type":"ContainerDied","Data":"9ee4dd3f8066e7176dfd18c0214a9b2dcbe912393950bd74336e785d9313dac3"} Jul 09 15:33:11.693083 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.693018 2577 scope.go:117] "RemoveContainer" containerID="14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849" Jul 09 15:33:11.703697 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.703674 2577 scope.go:117] "RemoveContainer" containerID="14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849" Jul 09 15:33:11.704007 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:33:11.703984 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849\": container with ID starting with 14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849 not found: ID does not exist" containerID="14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849" Jul 09 15:33:11.704093 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.704014 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849"} err="failed to get container status \"14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849\": rpc error: code = NotFound desc = could not find container \"14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849\": container with ID starting with 14e8aeb2cfed2592dbc26a3865d0596dfc8f9d23a82f4235428b664e689c5849 not found: ID does not exist" Jul 09 15:33:11.720761 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.720724 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c869bbbbb-rt2wm"] Jul 09 15:33:11.723433 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.723406 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-serving-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:33:11.723576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.723444 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62f47351-9497-41df-9f3c-4afccd76f9e1-console-oauth-config\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:33:11.723576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.723474 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-772hf\" (UniqueName: \"kubernetes.io/projected/62f47351-9497-41df-9f3c-4afccd76f9e1-kube-api-access-772hf\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:33:11.723576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.723488 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-console-config\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:33:11.723576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.723506 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-oauth-serving-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:33:11.723576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.723519 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62f47351-9497-41df-9f3c-4afccd76f9e1-service-ca\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:33:11.725651 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:11.725624 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c869bbbbb-rt2wm"] Jul 09 15:33:12.135526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:12.135493 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f47351-9497-41df-9f3c-4afccd76f9e1" path="/var/lib/kubelet/pods/62f47351-9497-41df-9f3c-4afccd76f9e1/volumes" Jul 09 15:33:13.704421 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:13.704383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"118de5ad4eb6c22b283c28087c49d552bc4ae03019e9182020c34c5b8f041a88"} Jul 09 15:33:13.704853 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:13.704450 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"04372c2e49d4233a10bafb61ca1033f64fa7c2e63184fb1c705edc2f406de0b2"} Jul 09 15:33:13.704853 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:13.704491 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"8869904b328f4db164578e695b5ce39077263d9de742581c7adac1a165c63062"} Jul 09 15:33:15.714827 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:15.714790 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"6387b70bdcd5b43ee64323d100191a3a320626d92c485c67bac7dd4fd6045562"} Jul 09 15:33:15.715243 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:15.714887 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"69e3b9af8d216a9a220152b9c9effa4cf94551526ba0a4fda81678fc41e3325a"} Jul 09 15:33:15.715243 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:15.714918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" event={"ID":"59ffaac6-e6df-43e4-b257-41d7f26c4312","Type":"ContainerStarted","Data":"2c374d7e238d7290edb0b2f8de7ac323b73fd388d0c9779c221af86a4e839eef"} Jul 09 15:33:15.715243 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:15.714961 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:33:15.739795 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:15.739738 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" podStartSLOduration=1.591529049 podStartE2EDuration="8.739716137s" totalImagesPullingTime="7.148187088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:15.738019502 +0000 UTC m=+112.275246677" watchObservedRunningTime="2026-07-09 15:33:15.739716137 +0000 UTC m=+112.276943312" Jul 09 15:33:18.995444 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:18.995399 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:18.995891 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:18.995488 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:19.000330 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:19.000307 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:19.733569 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:19.733542 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:33:21.724533 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:33:21.724503 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5bbd9b4c9-gfn4l" Jul 09 15:34:34.991994 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:34.991953 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6667dcdc57-2ltlj"] Jul 09 15:34:34.992681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:34.992380 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f47351-9497-41df-9f3c-4afccd76f9e1" containerName="console" Jul 09 15:34:34.992681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:34.992401 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f47351-9497-41df-9f3c-4afccd76f9e1" containerName="console" Jul 09 15:34:34.992681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:34.992507 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f47351-9497-41df-9f3c-4afccd76f9e1" containerName="console" Jul 09 15:34:34.995103 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:34.995081 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.008915 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.008885 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6667dcdc57-2ltlj"] Jul 09 15:34:35.123631 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-serving-cert\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.123818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-service-ca\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.123818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvbmt\" (UniqueName: \"kubernetes.io/projected/c35326e2-c2ef-4abf-a296-5291238514d9-kube-api-access-hvbmt\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.123818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-console-config\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.123818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123762 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-oauth-serving-cert\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.123818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123784 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-oauth-config\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.123977 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.123845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-trusted-ca-bundle\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224497 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-service-ca\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvbmt\" (UniqueName: \"kubernetes.io/projected/c35326e2-c2ef-4abf-a296-5291238514d9-kube-api-access-hvbmt\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-console-config\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-oauth-serving-cert\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224768 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-oauth-config\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224824 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224801 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-trusted-ca-bundle\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.224921 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.224865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-serving-cert\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.225274 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.225250 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-service-ca\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.225426 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.225402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-oauth-serving-cert\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.225547 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.225435 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-console-config\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.225637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.225617 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-trusted-ca-bundle\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.227146 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.227122 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-oauth-config\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.227243 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.227228 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-serving-cert\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.235162 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.235137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvbmt\" (UniqueName: \"kubernetes.io/projected/c35326e2-c2ef-4abf-a296-5291238514d9-kube-api-access-hvbmt\") pod \"console-6667dcdc57-2ltlj\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.304774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.304686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:35.440241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:35.440168 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6667dcdc57-2ltlj"] Jul 09 15:34:35.442823 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:34:35.442800 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35326e2_c2ef_4abf_a296_5291238514d9.slice/crio-869b5328d19f76746ac14dae28ba9caa330ded299321cf42b890368da2aebea8 WatchSource:0}: Error finding container 869b5328d19f76746ac14dae28ba9caa330ded299321cf42b890368da2aebea8: Status 404 returned error can't find the container with id 869b5328d19f76746ac14dae28ba9caa330ded299321cf42b890368da2aebea8 Jul 09 15:34:36.015773 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:36.015733 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6667dcdc57-2ltlj" event={"ID":"c35326e2-c2ef-4abf-a296-5291238514d9","Type":"ContainerStarted","Data":"5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf"} Jul 09 15:34:36.016145 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:36.015780 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6667dcdc57-2ltlj" event={"ID":"c35326e2-c2ef-4abf-a296-5291238514d9","Type":"ContainerStarted","Data":"869b5328d19f76746ac14dae28ba9caa330ded299321cf42b890368da2aebea8"} Jul 09 15:34:36.041601 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:36.041524 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6667dcdc57-2ltlj" podStartSLOduration=2.041507656 podStartE2EDuration="2.041507656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:36.03988822 +0000 UTC m=+192.577115393" watchObservedRunningTime="2026-07-09 15:34:36.041507656 +0000 UTC m=+192.578734828" Jul 09 15:34:40.000303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.000270 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6667dcdc57-2ltlj"] Jul 09 15:34:40.033043 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.033013 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d486fd6d8-fxfb6"] Jul 09 15:34:40.035568 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.035547 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.047285 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.047264 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d486fd6d8-fxfb6"] Jul 09 15:34:40.165788 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.165754 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-console-config\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.165788 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.165789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-trusted-ca-bundle\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.166005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.165818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796ee29c-723a-482e-865c-003658434a50-console-serving-cert\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.166005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.165864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-oauth-serving-cert\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.166005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.165915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796ee29c-723a-482e-865c-003658434a50-console-oauth-config\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.166005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.165967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-service-ca\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.166143 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.166006 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpzbk\" (UniqueName: \"kubernetes.io/projected/796ee29c-723a-482e-865c-003658434a50-kube-api-access-gpzbk\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.266660 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-service-ca\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.266660 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpzbk\" (UniqueName: \"kubernetes.io/projected/796ee29c-723a-482e-865c-003658434a50-kube-api-access-gpzbk\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.266878 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-console-config\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.266878 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266774 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-trusted-ca-bundle\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.266878 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266822 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796ee29c-723a-482e-865c-003658434a50-console-serving-cert\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.266878 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-oauth-serving-cert\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.267073 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.266901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796ee29c-723a-482e-865c-003658434a50-console-oauth-config\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.267418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.267388 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-console-config\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.267418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.267391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-service-ca\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.267657 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.267582 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-oauth-serving-cert\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.267774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.267753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796ee29c-723a-482e-865c-003658434a50-trusted-ca-bundle\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.269333 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.269308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796ee29c-723a-482e-865c-003658434a50-console-serving-cert\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.269458 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.269436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796ee29c-723a-482e-865c-003658434a50-console-oauth-config\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.277105 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.277079 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpzbk\" (UniqueName: \"kubernetes.io/projected/796ee29c-723a-482e-865c-003658434a50-kube-api-access-gpzbk\") pod \"console-5d486fd6d8-fxfb6\" (UID: \"796ee29c-723a-482e-865c-003658434a50\") " pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.345140 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.345100 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:40.478010 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:40.477977 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d486fd6d8-fxfb6"] Jul 09 15:34:40.481205 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:34:40.481178 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796ee29c_723a_482e_865c_003658434a50.slice/crio-fde31623282bec42e9a450db0a99e35cfa7b760e7284bec5f2693b9c51d9e164 WatchSource:0}: Error finding container fde31623282bec42e9a450db0a99e35cfa7b760e7284bec5f2693b9c51d9e164: Status 404 returned error can't find the container with id fde31623282bec42e9a450db0a99e35cfa7b760e7284bec5f2693b9c51d9e164 Jul 09 15:34:41.035802 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:41.035760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d486fd6d8-fxfb6" event={"ID":"796ee29c-723a-482e-865c-003658434a50","Type":"ContainerStarted","Data":"5c3db7fbd9a8513c92f9c89641cec4be79d377a4c2fc219061dd1c2e04f2889c"} Jul 09 15:34:41.036348 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:41.035842 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d486fd6d8-fxfb6" event={"ID":"796ee29c-723a-482e-865c-003658434a50","Type":"ContainerStarted","Data":"fde31623282bec42e9a450db0a99e35cfa7b760e7284bec5f2693b9c51d9e164"} Jul 09 15:34:41.054709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:41.054653 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d486fd6d8-fxfb6" podStartSLOduration=1.054636119 podStartE2EDuration="1.054636119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:41.053787028 +0000 UTC m=+197.591014203" watchObservedRunningTime="2026-07-09 15:34:41.054636119 +0000 UTC m=+197.591863297" Jul 09 15:34:45.305508 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:45.305437 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:34:50.346074 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:50.346040 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:50.346516 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:50.346104 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:50.350954 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:50.350927 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:51.076817 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:51.076787 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d486fd6d8-fxfb6" Jul 09 15:34:51.130662 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:34:51.130629 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7896b567-5fjq8"] Jul 09 15:35:05.019642 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.019578 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6667dcdc57-2ltlj" podUID="c35326e2-c2ef-4abf-a296-5291238514d9" containerName="console" containerID="cri-o://5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf" gracePeriod=15 Jul 09 15:35:05.263808 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.263785 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:35:05.371657 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371564 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-trusted-ca-bundle\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.371657 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371610 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-console-config\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.371657 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371632 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-oauth-serving-cert\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.371906 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371684 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-service-ca\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.371906 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371710 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvbmt\" (UniqueName: \"kubernetes.io/projected/c35326e2-c2ef-4abf-a296-5291238514d9-kube-api-access-hvbmt\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.371906 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371786 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-oauth-config\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.371906 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.371827 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-serving-cert\") pod \"c35326e2-c2ef-4abf-a296-5291238514d9\" (UID: \"c35326e2-c2ef-4abf-a296-5291238514d9\") " Jul 09 15:35:05.372110 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.372090 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-console-config" (OuterVolumeSpecName: "console-config") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:05.372253 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.372222 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:05.372253 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.372231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:05.372384 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.372254 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:05.374085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.374055 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:35:05.374190 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.374086 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:35:05.374190 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.374099 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c35326e2-c2ef-4abf-a296-5291238514d9-kube-api-access-hvbmt" (OuterVolumeSpecName: "kube-api-access-hvbmt") pod "c35326e2-c2ef-4abf-a296-5291238514d9" (UID: "c35326e2-c2ef-4abf-a296-5291238514d9"). InnerVolumeSpecName "kube-api-access-hvbmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:35:05.472592 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472534 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-service-ca\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:05.472592 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472588 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvbmt\" (UniqueName: \"kubernetes.io/projected/c35326e2-c2ef-4abf-a296-5291238514d9-kube-api-access-hvbmt\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:05.472592 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472599 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-oauth-config\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:05.472592 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472609 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c35326e2-c2ef-4abf-a296-5291238514d9-console-serving-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:05.472867 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472618 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-trusted-ca-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:05.472867 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472626 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-console-config\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:05.472867 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:05.472634 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c35326e2-c2ef-4abf-a296-5291238514d9-oauth-serving-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:06.130026 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.129989 2577 generic.go:356] "Generic (PLEG): container finished" podID="c35326e2-c2ef-4abf-a296-5291238514d9" containerID="5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf" exitCode=0 Jul 09 15:35:06.130489 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.130117 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6667dcdc57-2ltlj" Jul 09 15:35:06.135352 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.135324 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6667dcdc57-2ltlj" event={"ID":"c35326e2-c2ef-4abf-a296-5291238514d9","Type":"ContainerDied","Data":"5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf"} Jul 09 15:35:06.135530 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.135376 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6667dcdc57-2ltlj" event={"ID":"c35326e2-c2ef-4abf-a296-5291238514d9","Type":"ContainerDied","Data":"869b5328d19f76746ac14dae28ba9caa330ded299321cf42b890368da2aebea8"} Jul 09 15:35:06.135530 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.135435 2577 scope.go:117] "RemoveContainer" containerID="5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf" Jul 09 15:35:06.146852 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.146589 2577 scope.go:117] "RemoveContainer" containerID="5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf" Jul 09 15:35:06.146998 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:35:06.146979 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf\": container with ID starting with 5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf not found: ID does not exist" containerID="5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf" Jul 09 15:35:06.147063 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.147004 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf"} err="failed to get container status \"5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf\": rpc error: code = NotFound desc = could not find container \"5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf\": container with ID starting with 5554a7f9dd2c4bf84e5fec62f091d0b0bed3acdcd9dac815aeb5d2d60c04b9cf not found: ID does not exist" Jul 09 15:35:06.159508 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.159478 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6667dcdc57-2ltlj"] Jul 09 15:35:06.164403 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:06.164379 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6667dcdc57-2ltlj"] Jul 09 15:35:08.136011 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:08.135966 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c35326e2-c2ef-4abf-a296-5291238514d9" path="/var/lib/kubelet/pods/c35326e2-c2ef-4abf-a296-5291238514d9/volumes" Jul 09 15:35:16.152331 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.152287 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f7896b567-5fjq8" podUID="536ea1cf-3191-4713-8a32-e341388008cf" containerName="console" containerID="cri-o://855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f" gracePeriod=15 Jul 09 15:35:16.390592 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.390565 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:35:16.473719 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473637 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-service-ca\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.473719 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473683 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-oauth-serving-cert\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.473719 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473701 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-serving-cert\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.473719 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473721 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-console-config\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.473999 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473783 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-trusted-ca-bundle\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.473999 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473801 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxrrr\" (UniqueName: \"kubernetes.io/projected/536ea1cf-3191-4713-8a32-e341388008cf-kube-api-access-zxrrr\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.473999 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.473859 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-oauth-config\") pod \"536ea1cf-3191-4713-8a32-e341388008cf\" (UID: \"536ea1cf-3191-4713-8a32-e341388008cf\") " Jul 09 15:35:16.474178 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.474146 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-service-ca" (OuterVolumeSpecName: "service-ca") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:16.474241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.474151 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:16.474306 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.474291 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-console-config" (OuterVolumeSpecName: "console-config") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:16.474442 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.474413 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:35:16.475943 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.475910 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/536ea1cf-3191-4713-8a32-e341388008cf-kube-api-access-zxrrr" (OuterVolumeSpecName: "kube-api-access-zxrrr") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "kube-api-access-zxrrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:35:16.475943 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.475918 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:35:16.476068 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.475987 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "536ea1cf-3191-4713-8a32-e341388008cf" (UID: "536ea1cf-3191-4713-8a32-e341388008cf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:35:16.574665 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574630 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-trusted-ca-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:16.574665 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574660 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxrrr\" (UniqueName: \"kubernetes.io/projected/536ea1cf-3191-4713-8a32-e341388008cf-kube-api-access-zxrrr\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:16.574665 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574670 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-oauth-config\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:16.574898 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574680 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-service-ca\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:16.574898 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574689 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-oauth-serving-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:16.574898 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574698 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/536ea1cf-3191-4713-8a32-e341388008cf-console-serving-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:16.574898 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:16.574706 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/536ea1cf-3191-4713-8a32-e341388008cf-console-config\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:35:17.175230 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.175188 2577 generic.go:356] "Generic (PLEG): container finished" podID="536ea1cf-3191-4713-8a32-e341388008cf" containerID="855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f" exitCode=0 Jul 09 15:35:17.175654 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.175238 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7896b567-5fjq8" Jul 09 15:35:17.175654 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.175245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7896b567-5fjq8" event={"ID":"536ea1cf-3191-4713-8a32-e341388008cf","Type":"ContainerDied","Data":"855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f"} Jul 09 15:35:17.175654 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.175292 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7896b567-5fjq8" event={"ID":"536ea1cf-3191-4713-8a32-e341388008cf","Type":"ContainerDied","Data":"3075ea8befb9ada7b0a0177dc8ee8f22f10db42d4307c567da2bd29172183f42"} Jul 09 15:35:17.175654 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.175319 2577 scope.go:117] "RemoveContainer" containerID="855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f" Jul 09 15:35:17.183407 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.183383 2577 scope.go:117] "RemoveContainer" containerID="855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f" Jul 09 15:35:17.183703 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:35:17.183683 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f\": container with ID starting with 855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f not found: ID does not exist" containerID="855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f" Jul 09 15:35:17.183755 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.183716 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f"} err="failed to get container status \"855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f\": rpc error: code = NotFound desc = could not find container \"855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f\": container with ID starting with 855a52d98581798a7cd59fb138687bb1073cf96c484227551bf4afae5dbec23f not found: ID does not exist" Jul 09 15:35:17.205221 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.205190 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7896b567-5fjq8"] Jul 09 15:35:17.208007 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:17.207982 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f7896b567-5fjq8"] Jul 09 15:35:18.134614 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:18.134578 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="536ea1cf-3191-4713-8a32-e341388008cf" path="/var/lib/kubelet/pods/536ea1cf-3191-4713-8a32-e341388008cf/volumes" Jul 09 15:35:39.848536 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848503 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r"] Jul 09 15:35:39.848938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848812 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536ea1cf-3191-4713-8a32-e341388008cf" containerName="console" Jul 09 15:35:39.848938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848823 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="536ea1cf-3191-4713-8a32-e341388008cf" containerName="console" Jul 09 15:35:39.848938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848847 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c35326e2-c2ef-4abf-a296-5291238514d9" containerName="console" Jul 09 15:35:39.848938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848852 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c35326e2-c2ef-4abf-a296-5291238514d9" containerName="console" Jul 09 15:35:39.848938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848908 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="536ea1cf-3191-4713-8a32-e341388008cf" containerName="console" Jul 09 15:35:39.848938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.848914 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c35326e2-c2ef-4abf-a296-5291238514d9" containerName="console" Jul 09 15:35:39.853743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.853716 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.856098 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.856068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 15:35:39.856239 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.856112 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5s8hc\"" Jul 09 15:35:39.856487 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.856456 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 15:35:39.860883 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.860854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.861057 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.860951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcr4n\" (UniqueName: \"kubernetes.io/projected/e283190e-0144-48bd-a698-9df372eafa07-kube-api-access-mcr4n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.861057 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.861035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.863412 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.863384 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r"] Jul 09 15:35:39.961672 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.961635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcr4n\" (UniqueName: \"kubernetes.io/projected/e283190e-0144-48bd-a698-9df372eafa07-kube-api-access-mcr4n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.961672 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.961671 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.961886 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.961725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.962144 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.962121 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.962179 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.962128 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:39.971644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:39.971613 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcr4n\" (UniqueName: \"kubernetes.io/projected/e283190e-0144-48bd-a698-9df372eafa07-kube-api-access-mcr4n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:40.163678 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:40.163644 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:35:40.295969 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:40.295936 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r"] Jul 09 15:35:40.299357 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:35:40.299327 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode283190e_0144_48bd_a698_9df372eafa07.slice/crio-d0c35c22491093e2c154d0868ac12bf6dd207ea906c099ec66af878e448ce043 WatchSource:0}: Error finding container d0c35c22491093e2c154d0868ac12bf6dd207ea906c099ec66af878e448ce043: Status 404 returned error can't find the container with id d0c35c22491093e2c154d0868ac12bf6dd207ea906c099ec66af878e448ce043 Jul 09 15:35:41.265419 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:41.265382 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" event={"ID":"e283190e-0144-48bd-a698-9df372eafa07","Type":"ContainerStarted","Data":"d0c35c22491093e2c154d0868ac12bf6dd207ea906c099ec66af878e448ce043"} Jul 09 15:35:48.296938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:48.296905 2577 generic.go:356] "Generic (PLEG): container finished" podID="e283190e-0144-48bd-a698-9df372eafa07" containerID="91cba5eea1724d16acf60cd250dab7be70f5757103225c6e15b7022008e27801" exitCode=0 Jul 09 15:35:48.297318 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:48.296976 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" event={"ID":"e283190e-0144-48bd-a698-9df372eafa07","Type":"ContainerDied","Data":"91cba5eea1724d16acf60cd250dab7be70f5757103225c6e15b7022008e27801"} Jul 09 15:35:52.316433 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:52.316402 2577 generic.go:356] "Generic (PLEG): container finished" podID="e283190e-0144-48bd-a698-9df372eafa07" containerID="8ad51d2f576c42d3b86a3105e03dac6277662f440286a9750ede55c28d724071" exitCode=0 Jul 09 15:35:52.316818 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:35:52.316493 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" event={"ID":"e283190e-0144-48bd-a698-9df372eafa07","Type":"ContainerDied","Data":"8ad51d2f576c42d3b86a3105e03dac6277662f440286a9750ede55c28d724071"} Jul 09 15:36:00.351822 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:00.351790 2577 generic.go:356] "Generic (PLEG): container finished" podID="e283190e-0144-48bd-a698-9df372eafa07" containerID="cd6012331544ef89976309f20b086044957efd0e85560e6e0d127630d3ef5ba6" exitCode=0 Jul 09 15:36:00.352226 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:00.351838 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" event={"ID":"e283190e-0144-48bd-a698-9df372eafa07","Type":"ContainerDied","Data":"cd6012331544ef89976309f20b086044957efd0e85560e6e0d127630d3ef5ba6"} Jul 09 15:36:01.523119 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.523087 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:36:01.662180 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.662150 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-util\") pod \"e283190e-0144-48bd-a698-9df372eafa07\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " Jul 09 15:36:01.662338 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.662243 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-bundle\") pod \"e283190e-0144-48bd-a698-9df372eafa07\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " Jul 09 15:36:01.662338 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.662293 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcr4n\" (UniqueName: \"kubernetes.io/projected/e283190e-0144-48bd-a698-9df372eafa07-kube-api-access-mcr4n\") pod \"e283190e-0144-48bd-a698-9df372eafa07\" (UID: \"e283190e-0144-48bd-a698-9df372eafa07\") " Jul 09 15:36:01.662904 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.662882 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-bundle" (OuterVolumeSpecName: "bundle") pod "e283190e-0144-48bd-a698-9df372eafa07" (UID: "e283190e-0144-48bd-a698-9df372eafa07"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:36:01.664503 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.664479 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e283190e-0144-48bd-a698-9df372eafa07-kube-api-access-mcr4n" (OuterVolumeSpecName: "kube-api-access-mcr4n") pod "e283190e-0144-48bd-a698-9df372eafa07" (UID: "e283190e-0144-48bd-a698-9df372eafa07"). InnerVolumeSpecName "kube-api-access-mcr4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:36:01.666436 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.666416 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-util" (OuterVolumeSpecName: "util") pod "e283190e-0144-48bd-a698-9df372eafa07" (UID: "e283190e-0144-48bd-a698-9df372eafa07"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:36:01.763757 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.763724 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:01.763757 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.763751 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcr4n\" (UniqueName: \"kubernetes.io/projected/e283190e-0144-48bd-a698-9df372eafa07-kube-api-access-mcr4n\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:01.763757 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:01.763763 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e283190e-0144-48bd-a698-9df372eafa07-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:02.360886 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:02.360857 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" Jul 09 15:36:02.361046 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:02.360857 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404bg8r" event={"ID":"e283190e-0144-48bd-a698-9df372eafa07","Type":"ContainerDied","Data":"d0c35c22491093e2c154d0868ac12bf6dd207ea906c099ec66af878e448ce043"} Jul 09 15:36:02.361046 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:02.360973 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0c35c22491093e2c154d0868ac12bf6dd207ea906c099ec66af878e448ce043" Jul 09 15:36:07.857424 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857382 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x"] Jul 09 15:36:07.857877 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857833 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="util" Jul 09 15:36:07.857877 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857851 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="util" Jul 09 15:36:07.857877 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857869 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="extract" Jul 09 15:36:07.857877 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857875 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="extract" Jul 09 15:36:07.858004 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857885 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="pull" Jul 09 15:36:07.858004 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857890 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="pull" Jul 09 15:36:07.858004 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.857966 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e283190e-0144-48bd-a698-9df372eafa07" containerName="extract" Jul 09 15:36:07.860270 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.860253 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:07.863871 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.863850 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c52vv\"" Jul 09 15:36:07.864014 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.863999 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 09 15:36:07.864392 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.864378 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 09 15:36:07.879480 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:07.879435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x"] Jul 09 15:36:08.012427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.012392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnr76\" (UniqueName: \"kubernetes.io/projected/72bcff0d-cd5a-4616-b282-36391860a9eb-kube-api-access-lnr76\") pod \"cert-manager-operator-controller-manager-55c444bf86-qxs6x\" (UID: \"72bcff0d-cd5a-4616-b282-36391860a9eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.012427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.012430 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72bcff0d-cd5a-4616-b282-36391860a9eb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qxs6x\" (UID: \"72bcff0d-cd5a-4616-b282-36391860a9eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.113644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.113546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnr76\" (UniqueName: \"kubernetes.io/projected/72bcff0d-cd5a-4616-b282-36391860a9eb-kube-api-access-lnr76\") pod \"cert-manager-operator-controller-manager-55c444bf86-qxs6x\" (UID: \"72bcff0d-cd5a-4616-b282-36391860a9eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.113644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.113587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72bcff0d-cd5a-4616-b282-36391860a9eb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qxs6x\" (UID: \"72bcff0d-cd5a-4616-b282-36391860a9eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.113931 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.113915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72bcff0d-cd5a-4616-b282-36391860a9eb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qxs6x\" (UID: \"72bcff0d-cd5a-4616-b282-36391860a9eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.125516 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.125492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnr76\" (UniqueName: \"kubernetes.io/projected/72bcff0d-cd5a-4616-b282-36391860a9eb-kube-api-access-lnr76\") pod \"cert-manager-operator-controller-manager-55c444bf86-qxs6x\" (UID: \"72bcff0d-cd5a-4616-b282-36391860a9eb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.170027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.169994 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" Jul 09 15:36:08.309450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.309428 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x"] Jul 09 15:36:08.311998 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:36:08.311970 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bcff0d_cd5a_4616_b282_36391860a9eb.slice/crio-c1feec32245135c0f0bb5c87df7aa9acd1e980c004f8bb54f45f91e55ed5b16a WatchSource:0}: Error finding container c1feec32245135c0f0bb5c87df7aa9acd1e980c004f8bb54f45f91e55ed5b16a: Status 404 returned error can't find the container with id c1feec32245135c0f0bb5c87df7aa9acd1e980c004f8bb54f45f91e55ed5b16a Jul 09 15:36:08.388221 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:08.388185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" event={"ID":"72bcff0d-cd5a-4616-b282-36391860a9eb","Type":"ContainerStarted","Data":"c1feec32245135c0f0bb5c87df7aa9acd1e980c004f8bb54f45f91e55ed5b16a"} Jul 09 15:36:11.402936 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:11.402896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" event={"ID":"72bcff0d-cd5a-4616-b282-36391860a9eb","Type":"ContainerStarted","Data":"c33779c766f723370d314e1b268aa7707dee9708d5f982a58490513a8f5b66ec"} Jul 09 15:36:11.434302 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:11.434245 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qxs6x" podStartSLOduration=2.348928386 podStartE2EDuration="4.434213739s" totalImagesPullingTime="2.085285353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:36:11.432144371 +0000 UTC m=+287.969371546" watchObservedRunningTime="2026-07-09 15:36:11.434213739 +0000 UTC m=+287.971440914" Jul 09 15:36:12.393813 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.393777 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97"] Jul 09 15:36:12.396405 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.396384 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.399018 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.398999 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 15:36:12.399122 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.398999 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 15:36:12.399432 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.399417 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5s8hc\"" Jul 09 15:36:12.411015 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.410989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97"] Jul 09 15:36:12.552264 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.552214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.552452 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.552326 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.552452 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.552367 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn4xc\" (UniqueName: \"kubernetes.io/projected/d7841b98-b974-4ab9-a453-e7d0d837aa0c-kube-api-access-hn4xc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.653788 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.653699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.653788 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.653741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4xc\" (UniqueName: \"kubernetes.io/projected/d7841b98-b974-4ab9-a453-e7d0d837aa0c-kube-api-access-hn4xc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.653988 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.653796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.654112 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.654092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.654147 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.654117 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.663940 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.663909 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn4xc\" (UniqueName: \"kubernetes.io/projected/d7841b98-b974-4ab9-a453-e7d0d837aa0c-kube-api-access-hn4xc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.705918 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.705885 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:12.845017 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:12.844981 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97"] Jul 09 15:36:12.847703 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:36:12.847675 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7841b98_b974_4ab9_a453_e7d0d837aa0c.slice/crio-fa2118e0b74b7432dcb05bd5517a8cb5b56740310d3451c730fc1f8a674aa70f WatchSource:0}: Error finding container fa2118e0b74b7432dcb05bd5517a8cb5b56740310d3451c730fc1f8a674aa70f: Status 404 returned error can't find the container with id fa2118e0b74b7432dcb05bd5517a8cb5b56740310d3451c730fc1f8a674aa70f Jul 09 15:36:13.414007 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:13.413972 2577 generic.go:356] "Generic (PLEG): container finished" podID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerID="787021e6ee1f4bc76be486e07938176d686c7a1ac534364557a5a090f43fc5f8" exitCode=0 Jul 09 15:36:13.414388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:13.414063 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" event={"ID":"d7841b98-b974-4ab9-a453-e7d0d837aa0c","Type":"ContainerDied","Data":"787021e6ee1f4bc76be486e07938176d686c7a1ac534364557a5a090f43fc5f8"} Jul 09 15:36:13.414388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:13.414113 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" event={"ID":"d7841b98-b974-4ab9-a453-e7d0d837aa0c","Type":"ContainerStarted","Data":"fa2118e0b74b7432dcb05bd5517a8cb5b56740310d3451c730fc1f8a674aa70f"} Jul 09 15:36:15.425348 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:15.425308 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" event={"ID":"d7841b98-b974-4ab9-a453-e7d0d837aa0c","Type":"ContainerStarted","Data":"9d4960396b6f5c44349a47d107a4c02395b90dbccb563c6df078ca1b2190d1d3"} Jul 09 15:36:16.431408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:16.431372 2577 generic.go:356] "Generic (PLEG): container finished" podID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerID="9d4960396b6f5c44349a47d107a4c02395b90dbccb563c6df078ca1b2190d1d3" exitCode=0 Jul 09 15:36:16.431829 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:16.431456 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" event={"ID":"d7841b98-b974-4ab9-a453-e7d0d837aa0c","Type":"ContainerDied","Data":"9d4960396b6f5c44349a47d107a4c02395b90dbccb563c6df078ca1b2190d1d3"} Jul 09 15:36:17.437728 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:17.437695 2577 generic.go:356] "Generic (PLEG): container finished" podID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerID="93d67dbdc3640dbdebd63cc150ccdb04ad6ff34e48606bcb76cffaf53da3491f" exitCode=0 Jul 09 15:36:17.438186 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:17.437755 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" event={"ID":"d7841b98-b974-4ab9-a453-e7d0d837aa0c","Type":"ContainerDied","Data":"93d67dbdc3640dbdebd63cc150ccdb04ad6ff34e48606bcb76cffaf53da3491f"} Jul 09 15:36:18.566392 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.566360 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:18.708529 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.708411 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn4xc\" (UniqueName: \"kubernetes.io/projected/d7841b98-b974-4ab9-a453-e7d0d837aa0c-kube-api-access-hn4xc\") pod \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " Jul 09 15:36:18.708669 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.708539 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-util\") pod \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " Jul 09 15:36:18.708669 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.708568 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-bundle\") pod \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\" (UID: \"d7841b98-b974-4ab9-a453-e7d0d837aa0c\") " Jul 09 15:36:18.709039 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.709001 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-bundle" (OuterVolumeSpecName: "bundle") pod "d7841b98-b974-4ab9-a453-e7d0d837aa0c" (UID: "d7841b98-b974-4ab9-a453-e7d0d837aa0c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:36:18.710688 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.710659 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7841b98-b974-4ab9-a453-e7d0d837aa0c-kube-api-access-hn4xc" (OuterVolumeSpecName: "kube-api-access-hn4xc") pod "d7841b98-b974-4ab9-a453-e7d0d837aa0c" (UID: "d7841b98-b974-4ab9-a453-e7d0d837aa0c"). InnerVolumeSpecName "kube-api-access-hn4xc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:36:18.713148 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.713113 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-util" (OuterVolumeSpecName: "util") pod "d7841b98-b974-4ab9-a453-e7d0d837aa0c" (UID: "d7841b98-b974-4ab9-a453-e7d0d837aa0c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:36:18.809207 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.809150 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn4xc\" (UniqueName: \"kubernetes.io/projected/d7841b98-b974-4ab9-a453-e7d0d837aa0c-kube-api-access-hn4xc\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:18.809207 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.809196 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:18.809207 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:18.809205 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7841b98-b974-4ab9-a453-e7d0d837aa0c-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:19.448644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:19.448604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" event={"ID":"d7841b98-b974-4ab9-a453-e7d0d837aa0c","Type":"ContainerDied","Data":"fa2118e0b74b7432dcb05bd5517a8cb5b56740310d3451c730fc1f8a674aa70f"} Jul 09 15:36:19.448644 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:19.448646 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fz7r97" Jul 09 15:36:19.448842 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:19.448666 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa2118e0b74b7432dcb05bd5517a8cb5b56740310d3451c730fc1f8a674aa70f" Jul 09 15:36:22.757595 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757557 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wptx5"] Jul 09 15:36:22.757997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757897 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="extract" Jul 09 15:36:22.757997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757910 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="extract" Jul 09 15:36:22.757997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757923 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="util" Jul 09 15:36:22.757997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757929 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="util" Jul 09 15:36:22.757997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757935 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="pull" Jul 09 15:36:22.757997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.757943 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="pull" Jul 09 15:36:22.758212 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.758006 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7841b98-b974-4ab9-a453-e7d0d837aa0c" containerName="extract" Jul 09 15:36:22.760457 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.760436 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:22.762777 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.762756 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 15:36:22.762777 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.762778 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 15:36:22.763493 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.763450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n57x2\"" Jul 09 15:36:22.779363 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.779331 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wptx5"] Jul 09 15:36:22.843183 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.843141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwldn\" (UniqueName: \"kubernetes.io/projected/657fa41f-c8c3-4609-ae60-fefd1dfc8158-kube-api-access-cwldn\") pod \"cert-manager-55b4d954d5-wptx5\" (UID: \"657fa41f-c8c3-4609-ae60-fefd1dfc8158\") " pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:22.843351 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.843192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/657fa41f-c8c3-4609-ae60-fefd1dfc8158-bound-sa-token\") pod \"cert-manager-55b4d954d5-wptx5\" (UID: \"657fa41f-c8c3-4609-ae60-fefd1dfc8158\") " pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:22.944683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.944646 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwldn\" (UniqueName: \"kubernetes.io/projected/657fa41f-c8c3-4609-ae60-fefd1dfc8158-kube-api-access-cwldn\") pod \"cert-manager-55b4d954d5-wptx5\" (UID: \"657fa41f-c8c3-4609-ae60-fefd1dfc8158\") " pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:22.944861 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.944705 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/657fa41f-c8c3-4609-ae60-fefd1dfc8158-bound-sa-token\") pod \"cert-manager-55b4d954d5-wptx5\" (UID: \"657fa41f-c8c3-4609-ae60-fefd1dfc8158\") " pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:22.957793 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.957760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/657fa41f-c8c3-4609-ae60-fefd1dfc8158-bound-sa-token\") pod \"cert-manager-55b4d954d5-wptx5\" (UID: \"657fa41f-c8c3-4609-ae60-fefd1dfc8158\") " pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:22.958037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:22.958016 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwldn\" (UniqueName: \"kubernetes.io/projected/657fa41f-c8c3-4609-ae60-fefd1dfc8158-kube-api-access-cwldn\") pod \"cert-manager-55b4d954d5-wptx5\" (UID: \"657fa41f-c8c3-4609-ae60-fefd1dfc8158\") " pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:23.078330 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:23.078235 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wptx5" Jul 09 15:36:23.219862 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:23.219218 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wptx5"] Jul 09 15:36:23.223660 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:36:23.223613 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod657fa41f_c8c3_4609_ae60_fefd1dfc8158.slice/crio-1d7221fb4d085bf04465fe5cafcc62353fcdf02cd72f7cfa47786ff12207b323 WatchSource:0}: Error finding container 1d7221fb4d085bf04465fe5cafcc62353fcdf02cd72f7cfa47786ff12207b323: Status 404 returned error can't find the container with id 1d7221fb4d085bf04465fe5cafcc62353fcdf02cd72f7cfa47786ff12207b323 Jul 09 15:36:23.467427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:23.467392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wptx5" event={"ID":"657fa41f-c8c3-4609-ae60-fefd1dfc8158","Type":"ContainerStarted","Data":"1d7221fb4d085bf04465fe5cafcc62353fcdf02cd72f7cfa47786ff12207b323"} Jul 09 15:36:23.954405 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:23.954335 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:36:23.959016 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:23.958993 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:36:23.961229 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:23.961204 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 15:36:26.485699 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:26.485601 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wptx5" event={"ID":"657fa41f-c8c3-4609-ae60-fefd1dfc8158","Type":"ContainerStarted","Data":"a0d0b353f668566e48cc16f1e4276719dee7d5815f835996848489a3ce2e6454"} Jul 09 15:36:26.506476 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:26.506399 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wptx5" podStartSLOduration=1.559209464 podStartE2EDuration="4.506379536s" totalImagesPullingTime="2.947170072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:36:26.505103775 +0000 UTC m=+303.042330963" watchObservedRunningTime="2026-07-09 15:36:26.506379536 +0000 UTC m=+303.043606711" Jul 09 15:36:31.400415 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.400379 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx"] Jul 09 15:36:31.403491 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.403454 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.407348 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.407331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.407413 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.407364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.407493 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.407472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fbt6\" (UniqueName: \"kubernetes.io/projected/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-kube-api-access-9fbt6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.407861 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.407848 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 15:36:31.408277 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.408262 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 15:36:31.408317 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.408270 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5s8hc\"" Jul 09 15:36:31.417538 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.417518 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx"] Jul 09 15:36:31.508745 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.508714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.508932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.508756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.508932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.508813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fbt6\" (UniqueName: \"kubernetes.io/projected/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-kube-api-access-9fbt6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.509148 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.509127 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.509211 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.509179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.520271 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.520245 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fbt6\" (UniqueName: \"kubernetes.io/projected/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-kube-api-access-9fbt6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.712947 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.712854 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:31.843789 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:31.843767 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx"] Jul 09 15:36:31.845549 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:36:31.845523 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2d68c9_22ae_4216_abdf_f7f08b3f957a.slice/crio-1885a902ef007dc8813316ad48d04f5f3bcdfcdaf0b90a6bb411fe17b626f38c WatchSource:0}: Error finding container 1885a902ef007dc8813316ad48d04f5f3bcdfcdaf0b90a6bb411fe17b626f38c: Status 404 returned error can't find the container with id 1885a902ef007dc8813316ad48d04f5f3bcdfcdaf0b90a6bb411fe17b626f38c Jul 09 15:36:32.511166 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:32.511130 2577 generic.go:356] "Generic (PLEG): container finished" podID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerID="919fe207a92cb0a1c60cc9fe499dbadb269fab0a2ee774f4f9493f761aa4c0e1" exitCode=0 Jul 09 15:36:32.511563 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:32.511225 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" event={"ID":"9d2d68c9-22ae-4216-abdf-f7f08b3f957a","Type":"ContainerDied","Data":"919fe207a92cb0a1c60cc9fe499dbadb269fab0a2ee774f4f9493f761aa4c0e1"} Jul 09 15:36:32.511563 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:32.511278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" event={"ID":"9d2d68c9-22ae-4216-abdf-f7f08b3f957a","Type":"ContainerStarted","Data":"1885a902ef007dc8813316ad48d04f5f3bcdfcdaf0b90a6bb411fe17b626f38c"} Jul 09 15:36:32.511750 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:32.511736 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:36:33.517365 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:33.517335 2577 generic.go:356] "Generic (PLEG): container finished" podID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerID="4b72f579fb0930836452f1b8e7a1d6245ed6599ce12a1bc9f264e9cb2674c98c" exitCode=0 Jul 09 15:36:33.517807 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:33.517408 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" event={"ID":"9d2d68c9-22ae-4216-abdf-f7f08b3f957a","Type":"ContainerDied","Data":"4b72f579fb0930836452f1b8e7a1d6245ed6599ce12a1bc9f264e9cb2674c98c"} Jul 09 15:36:34.524372 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:34.524334 2577 generic.go:356] "Generic (PLEG): container finished" podID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerID="8cdb52c9cfd58a8a349a8c6895ed308733a3c4b8fde1bb1e7698027821abd098" exitCode=0 Jul 09 15:36:34.524789 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:34.524420 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" event={"ID":"9d2d68c9-22ae-4216-abdf-f7f08b3f957a","Type":"ContainerDied","Data":"8cdb52c9cfd58a8a349a8c6895ed308733a3c4b8fde1bb1e7698027821abd098"} Jul 09 15:36:35.661435 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.661412 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:35.736290 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.736254 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-util\") pod \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " Jul 09 15:36:35.736498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.736300 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-bundle\") pod \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " Jul 09 15:36:35.736498 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.736325 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fbt6\" (UniqueName: \"kubernetes.io/projected/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-kube-api-access-9fbt6\") pod \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\" (UID: \"9d2d68c9-22ae-4216-abdf-f7f08b3f957a\") " Jul 09 15:36:35.737031 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.737009 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-bundle" (OuterVolumeSpecName: "bundle") pod "9d2d68c9-22ae-4216-abdf-f7f08b3f957a" (UID: "9d2d68c9-22ae-4216-abdf-f7f08b3f957a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:36:35.738445 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.738420 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-kube-api-access-9fbt6" (OuterVolumeSpecName: "kube-api-access-9fbt6") pod "9d2d68c9-22ae-4216-abdf-f7f08b3f957a" (UID: "9d2d68c9-22ae-4216-abdf-f7f08b3f957a"). InnerVolumeSpecName "kube-api-access-9fbt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:36:35.741968 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.741923 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-util" (OuterVolumeSpecName: "util") pod "9d2d68c9-22ae-4216-abdf-f7f08b3f957a" (UID: "9d2d68c9-22ae-4216-abdf-f7f08b3f957a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:36:35.836930 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.836841 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fbt6\" (UniqueName: \"kubernetes.io/projected/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-kube-api-access-9fbt6\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:35.836930 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.836875 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:35.836930 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:35.836885 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d68c9-22ae-4216-abdf-f7f08b3f957a-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:36:36.540324 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:36.540243 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" Jul 09 15:36:36.540324 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:36.540255 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7p4wx" event={"ID":"9d2d68c9-22ae-4216-abdf-f7f08b3f957a","Type":"ContainerDied","Data":"1885a902ef007dc8813316ad48d04f5f3bcdfcdaf0b90a6bb411fe17b626f38c"} Jul 09 15:36:36.540324 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:36.540295 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1885a902ef007dc8813316ad48d04f5f3bcdfcdaf0b90a6bb411fe17b626f38c" Jul 09 15:36:48.944981 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.944949 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf"] Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945262 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="util" Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945273 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="util" Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945287 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="extract" Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945292 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="extract" Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945302 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="pull" Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945308 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="pull" Jul 09 15:36:48.945450 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.945385 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2d68c9-22ae-4216-abdf-f7f08b3f957a" containerName="extract" Jul 09 15:36:48.947943 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.947921 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:48.950027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.950002 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 15:36:48.950143 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.950066 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 15:36:48.950427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.950412 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 15:36:48.950534 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.950516 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 15:36:48.950622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.950587 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-44wlf\"" Jul 09 15:36:48.962499 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:48.962475 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf"] Jul 09 15:36:49.037984 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.037945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/814184a8-c96c-4647-a69a-d9355265e3f4-manifests\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.037984 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.037984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/814184a8-c96c-4647-a69a-d9355265e3f4-tmp\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.038192 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.038005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/814184a8-c96c-4647-a69a-d9355265e3f4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.038192 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.038091 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p5l2\" (UniqueName: \"kubernetes.io/projected/814184a8-c96c-4647-a69a-d9355265e3f4-kube-api-access-7p5l2\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.038192 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.038124 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/814184a8-c96c-4647-a69a-d9355265e3f4-webhook-cert\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.139055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.139018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p5l2\" (UniqueName: \"kubernetes.io/projected/814184a8-c96c-4647-a69a-d9355265e3f4-kube-api-access-7p5l2\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.139055 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.139058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/814184a8-c96c-4647-a69a-d9355265e3f4-webhook-cert\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.139334 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.139141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/814184a8-c96c-4647-a69a-d9355265e3f4-manifests\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.139334 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.139166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/814184a8-c96c-4647-a69a-d9355265e3f4-tmp\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.139334 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.139191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/814184a8-c96c-4647-a69a-d9355265e3f4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.139683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.139656 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/814184a8-c96c-4647-a69a-d9355265e3f4-manifests\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.141419 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.141389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/814184a8-c96c-4647-a69a-d9355265e3f4-tmp\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.141550 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.141478 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/814184a8-c96c-4647-a69a-d9355265e3f4-webhook-cert\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.141550 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.141507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/814184a8-c96c-4647-a69a-d9355265e3f4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.150261 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.150238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p5l2\" (UniqueName: \"kubernetes.io/projected/814184a8-c96c-4647-a69a-d9355265e3f4-kube-api-access-7p5l2\") pod \"opendatahub-operator-controller-manager-9bb557cf6-zrptf\" (UID: \"814184a8-c96c-4647-a69a-d9355265e3f4\") " pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.260183 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.260102 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:36:49.405168 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.405139 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf"] Jul 09 15:36:49.406776 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:36:49.406749 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814184a8_c96c_4647_a69a_d9355265e3f4.slice/crio-9d533cb224dc8c40d4631f1d18e94d6d2ea2282c8a9740754f4e33ffe4a1891a WatchSource:0}: Error finding container 9d533cb224dc8c40d4631f1d18e94d6d2ea2282c8a9740754f4e33ffe4a1891a: Status 404 returned error can't find the container with id 9d533cb224dc8c40d4631f1d18e94d6d2ea2282c8a9740754f4e33ffe4a1891a Jul 09 15:36:49.604347 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:49.604259 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" event={"ID":"814184a8-c96c-4647-a69a-d9355265e3f4","Type":"ContainerStarted","Data":"9d533cb224dc8c40d4631f1d18e94d6d2ea2282c8a9740754f4e33ffe4a1891a"} Jul 09 15:36:52.622090 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:52.622058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" event={"ID":"814184a8-c96c-4647-a69a-d9355265e3f4","Type":"ContainerStarted","Data":"32790c0023e442e5bfb9091d3a78403a232c6ce848ff6e77de72ac0d9f322387"} Jul 09 15:36:53.627584 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:53.627554 2577 generic.go:356] "Generic (PLEG): container finished" podID="814184a8-c96c-4647-a69a-d9355265e3f4" containerID="32790c0023e442e5bfb9091d3a78403a232c6ce848ff6e77de72ac0d9f322387" exitCode=0 Jul 09 15:36:53.627936 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:36:53.627626 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" event={"ID":"814184a8-c96c-4647-a69a-d9355265e3f4","Type":"ContainerDied","Data":"32790c0023e442e5bfb9091d3a78403a232c6ce848ff6e77de72ac0d9f322387"} Jul 09 15:36:53.861447 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:36:53.861353 2577 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" image="quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2" Jul 09 15:36:53.861636 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:36:53.861490 2577 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod opendatahub-operator-controller-manager-9bb557cf6-zrptf_opendatahub(814184a8-c96c-4647-a69a-d9355265e3f4): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 09 15:36:53.861636 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:36:53.861525 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" podUID="814184a8-c96c-4647-a69a-d9355265e3f4" Jul 09 15:36:54.632892 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:36:54.632853 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" podUID="814184a8-c96c-4647-a69a-d9355265e3f4" Jul 09 15:37:10.701363 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:10.701325 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" event={"ID":"814184a8-c96c-4647-a69a-d9355265e3f4","Type":"ContainerStarted","Data":"4a211da6edb6c27966f8325e10bd5e3f7427deb0f44cec4caae3554c2f75d270"} Jul 09 15:37:10.701742 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:10.701542 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:37:10.726451 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:10.726371 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" podStartSLOduration=1.821382088 podStartE2EDuration="22.726357277s" totalImagesPullingTime="19.882370893s" totalInitContainerRuntime="1.022604296s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-09 15:37:10.724110724 +0000 UTC m=+347.261337899" watchObservedRunningTime="2026-07-09 15:37:10.726357277 +0000 UTC m=+347.263584451" Jul 09 15:37:21.708399 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:21.708366 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9bb557cf6-zrptf" Jul 09 15:37:24.339634 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.339601 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57f9565468-khrgp"] Jul 09 15:37:24.343275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.343256 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.346333 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.346314 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 15:37:24.346689 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.346667 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 15:37:24.346811 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.346795 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wlwsq\"" Jul 09 15:37:24.365583 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.365560 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57f9565468-khrgp"] Jul 09 15:37:24.447684 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.447640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b0aa18-600f-4596-a181-670b4d1d3c02-tls-certs\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.447861 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.447694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95b0aa18-600f-4596-a181-670b4d1d3c02-tmp\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.447861 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.447783 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w46r6\" (UniqueName: \"kubernetes.io/projected/95b0aa18-600f-4596-a181-670b4d1d3c02-kube-api-access-w46r6\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.480607 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.480575 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r9mvq"] Jul 09 15:37:24.483313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.483294 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:24.485426 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.485364 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 15:37:24.485729 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.485712 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hzfm9\"" Jul 09 15:37:24.494414 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.494385 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r9mvq"] Jul 09 15:37:24.548258 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.548229 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w46r6\" (UniqueName: \"kubernetes.io/projected/95b0aa18-600f-4596-a181-670b4d1d3c02-kube-api-access-w46r6\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.548420 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.548288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b0aa18-600f-4596-a181-670b4d1d3c02-tls-certs\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.548420 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.548311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95b0aa18-600f-4596-a181-670b4d1d3c02-tmp\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.550573 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.550544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95b0aa18-600f-4596-a181-670b4d1d3c02-tmp\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.550704 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.550606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b0aa18-600f-4596-a181-670b4d1d3c02-tls-certs\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.563123 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.563098 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w46r6\" (UniqueName: \"kubernetes.io/projected/95b0aa18-600f-4596-a181-670b4d1d3c02-kube-api-access-w46r6\") pod \"kube-auth-proxy-57f9565468-khrgp\" (UID: \"95b0aa18-600f-4596-a181-670b4d1d3c02\") " pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.649664 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.649634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p4lr\" (UniqueName: \"kubernetes.io/projected/27a197e8-3ab4-428d-9a43-9672a5087c60-kube-api-access-6p4lr\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:24.649843 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.649693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a197e8-3ab4-428d-9a43-9672a5087c60-cert\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:24.653559 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.653533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" Jul 09 15:37:24.751045 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.751009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p4lr\" (UniqueName: \"kubernetes.io/projected/27a197e8-3ab4-428d-9a43-9672a5087c60-kube-api-access-6p4lr\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:24.751213 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.751074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a197e8-3ab4-428d-9a43-9672a5087c60-cert\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:24.751213 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:37:24.751190 2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 15:37:24.751331 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:37:24.751252 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a197e8-3ab4-428d-9a43-9672a5087c60-cert podName:27a197e8-3ab4-428d-9a43-9672a5087c60 nodeName:}" failed. No retries permitted until 2026-07-09 15:37:25.251232509 +0000 UTC m=+361.788459669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/27a197e8-3ab4-428d-9a43-9672a5087c60-cert") pod "odh-model-controller-7998c67dc4-r9mvq" (UID: "27a197e8-3ab4-428d-9a43-9672a5087c60") : secret "odh-model-controller-webhook-cert" not found Jul 09 15:37:24.760045 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.760009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p4lr\" (UniqueName: \"kubernetes.io/projected/27a197e8-3ab4-428d-9a43-9672a5087c60-kube-api-access-6p4lr\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:24.787655 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:24.787632 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57f9565468-khrgp"] Jul 09 15:37:24.788896 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:37:24.788869 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b0aa18_600f_4596_a181_670b4d1d3c02.slice/crio-b4e839a3cb571983b49bd2171cbc54c450a66214f40dcb56fc60bcf91546a66b WatchSource:0}: Error finding container b4e839a3cb571983b49bd2171cbc54c450a66214f40dcb56fc60bcf91546a66b: Status 404 returned error can't find the container with id b4e839a3cb571983b49bd2171cbc54c450a66214f40dcb56fc60bcf91546a66b Jul 09 15:37:25.255320 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:25.255285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a197e8-3ab4-428d-9a43-9672a5087c60-cert\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:25.257691 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:25.257669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a197e8-3ab4-428d-9a43-9672a5087c60-cert\") pod \"odh-model-controller-7998c67dc4-r9mvq\" (UID: \"27a197e8-3ab4-428d-9a43-9672a5087c60\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:25.394984 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:25.394948 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:25.552799 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:25.552777 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r9mvq"] Jul 09 15:37:25.554588 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:37:25.554563 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a197e8_3ab4_428d_9a43_9672a5087c60.slice/crio-10ccfe70cb9845980c3b91d2835f21c8a174b3a8f391b9f159ede376caae81e8 WatchSource:0}: Error finding container 10ccfe70cb9845980c3b91d2835f21c8a174b3a8f391b9f159ede376caae81e8: Status 404 returned error can't find the container with id 10ccfe70cb9845980c3b91d2835f21c8a174b3a8f391b9f159ede376caae81e8 Jul 09 15:37:25.773196 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:25.773130 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" event={"ID":"27a197e8-3ab4-428d-9a43-9672a5087c60","Type":"ContainerStarted","Data":"10ccfe70cb9845980c3b91d2835f21c8a174b3a8f391b9f159ede376caae81e8"} Jul 09 15:37:25.774688 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:25.774654 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" event={"ID":"95b0aa18-600f-4596-a181-670b4d1d3c02","Type":"ContainerStarted","Data":"b4e839a3cb571983b49bd2171cbc54c450a66214f40dcb56fc60bcf91546a66b"} Jul 09 15:37:26.010417 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.010368 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv"] Jul 09 15:37:26.023983 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.023950 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv"] Jul 09 15:37:26.024157 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.024090 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.027443 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.027421 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 15:37:26.027607 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.027583 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 15:37:26.027690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.027420 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 15:37:26.027854 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.027827 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 15:37:26.027946 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.027869 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6qz8f\"" Jul 09 15:37:26.162131 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.162303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.162303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162170 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-cacerts\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.162303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kw4d\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-kube-api-access-7kw4d\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.162303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.162499 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162334 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-local-certs\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.162499 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.162377 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-token\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.263683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-token\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.263683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.263683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.263683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-cacerts\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.264065 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kw4d\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-kube-api-access-7kw4d\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.264065 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.264065 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.263782 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-local-certs\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.264537 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.264507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.266081 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.266054 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-local-certs\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.266081 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.266077 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-cacerts\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.266318 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.266298 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.266377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.266321 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.274198 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.274172 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-token\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.274599 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.274577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kw4d\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-kube-api-access-7kw4d\") pod \"istiod-openshift-gateway-7574459656-zw7xv\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.336327 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.336293 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:26.524640 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.524601 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv"] Jul 09 15:37:26.528847 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:37:26.528804 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5c088b_4e30_4ff6_9480_f8ca6b10c506.slice/crio-a379a0f6351debb7431d791b4dd1c264e3c00f538f2d9102cc465a86b5f168bb WatchSource:0}: Error finding container a379a0f6351debb7431d791b4dd1c264e3c00f538f2d9102cc465a86b5f168bb: Status 404 returned error can't find the container with id a379a0f6351debb7431d791b4dd1c264e3c00f538f2d9102cc465a86b5f168bb Jul 09 15:37:26.785520 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:26.785411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" event={"ID":"dc5c088b-4e30-4ff6-9480-f8ca6b10c506","Type":"ContainerStarted","Data":"a379a0f6351debb7431d791b4dd1c264e3c00f538f2d9102cc465a86b5f168bb"} Jul 09 15:37:30.212599 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.212561 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:37:30.212945 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.212635 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:37:30.811663 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.811622 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" event={"ID":"95b0aa18-600f-4596-a181-670b4d1d3c02","Type":"ContainerStarted","Data":"622a09829931030b89c45072e392e28f8777e14a35c81b0a46b29c78ad7e9ad5"} Jul 09 15:37:30.813098 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.813052 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" event={"ID":"27a197e8-3ab4-428d-9a43-9672a5087c60","Type":"ContainerStarted","Data":"cab141aaf0e674a414fcde66873f411d0ce0a51be9f13d0f2ed5f3899269c6f9"} Jul 09 15:37:30.813236 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.813125 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:37:30.814552 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.814530 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" event={"ID":"dc5c088b-4e30-4ff6-9480-f8ca6b10c506","Type":"ContainerStarted","Data":"d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39"} Jul 09 15:37:30.814665 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.814639 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:30.839124 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.839060 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57f9565468-khrgp" podStartSLOduration=1.463631194 podStartE2EDuration="6.839039557s" totalImagesPullingTime="5.375408363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:30.836609695 +0000 UTC m=+367.373836885" watchObservedRunningTime="2026-07-09 15:37:30.839039557 +0000 UTC m=+367.376266732" Jul 09 15:37:30.861774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.861713 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" podStartSLOduration=2.216678435 podStartE2EDuration="6.861697609s" totalImagesPullingTime="4.645019174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:30.859410988 +0000 UTC m=+367.396638162" watchObservedRunningTime="2026-07-09 15:37:30.861697609 +0000 UTC m=+367.398924783" Jul 09 15:37:30.885586 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:30.885535 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" podStartSLOduration=2.205027308 podStartE2EDuration="5.88551435s" totalImagesPullingTime="3.680487042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:30.883677754 +0000 UTC m=+367.420904930" watchObservedRunningTime="2026-07-09 15:37:30.88551435 +0000 UTC m=+367.422741526" Jul 09 15:37:31.821634 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:31.821587 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:36.178918 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.178879 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s"] Jul 09 15:37:36.181886 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.181863 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.185730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.185703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dt2d7\"" Jul 09 15:37:36.185892 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.185703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 15:37:36.196481 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.196428 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s"] Jul 09 15:37:36.221572 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.221541 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn"] Jul 09 15:37:36.224712 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.224691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.227964 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.227926 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vpgb9\"" Jul 09 15:37:36.228221 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.228198 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 15:37:36.234833 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.234802 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn"] Jul 09 15:37:36.256204 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.256181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dffc73f-6771-46be-a7ba-ab6b22d45cd5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ghsxn\" (UID: \"9dffc73f-6771-46be-a7ba-ab6b22d45cd5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.256344 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.256236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbc21ba3-7820-4cb8-b921-cb9f8072497a-cert\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.256344 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.256255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgsrz\" (UniqueName: \"kubernetes.io/projected/cbc21ba3-7820-4cb8-b921-cb9f8072497a-kube-api-access-jgsrz\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.256344 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.256290 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rznmn\" (UniqueName: \"kubernetes.io/projected/9dffc73f-6771-46be-a7ba-ab6b22d45cd5-kube-api-access-rznmn\") pod \"llmisvc-controller-manager-777c6bd7d5-ghsxn\" (UID: \"9dffc73f-6771-46be-a7ba-ab6b22d45cd5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.357751 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.357718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dffc73f-6771-46be-a7ba-ab6b22d45cd5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ghsxn\" (UID: \"9dffc73f-6771-46be-a7ba-ab6b22d45cd5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.357927 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.357773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbc21ba3-7820-4cb8-b921-cb9f8072497a-cert\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.357927 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.357790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgsrz\" (UniqueName: \"kubernetes.io/projected/cbc21ba3-7820-4cb8-b921-cb9f8072497a-kube-api-access-jgsrz\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.357927 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.357808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rznmn\" (UniqueName: \"kubernetes.io/projected/9dffc73f-6771-46be-a7ba-ab6b22d45cd5-kube-api-access-rznmn\") pod \"llmisvc-controller-manager-777c6bd7d5-ghsxn\" (UID: \"9dffc73f-6771-46be-a7ba-ab6b22d45cd5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.357927 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:37:36.357911 2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 09 15:37:36.358125 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:37:36.357996 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc21ba3-7820-4cb8-b921-cb9f8072497a-cert podName:cbc21ba3-7820-4cb8-b921-cb9f8072497a nodeName:}" failed. No retries permitted until 2026-07-09 15:37:36.85797712 +0000 UTC m=+373.395204275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cbc21ba3-7820-4cb8-b921-cb9f8072497a-cert") pod "kserve-controller-manager-d6f4b65b8-xwc9s" (UID: "cbc21ba3-7820-4cb8-b921-cb9f8072497a") : secret "kserve-webhook-server-cert" not found Jul 09 15:37:36.359923 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.359905 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dffc73f-6771-46be-a7ba-ab6b22d45cd5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ghsxn\" (UID: \"9dffc73f-6771-46be-a7ba-ab6b22d45cd5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.373898 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.373867 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgsrz\" (UniqueName: \"kubernetes.io/projected/cbc21ba3-7820-4cb8-b921-cb9f8072497a-kube-api-access-jgsrz\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.389072 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.389039 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rznmn\" (UniqueName: \"kubernetes.io/projected/9dffc73f-6771-46be-a7ba-ab6b22d45cd5-kube-api-access-rznmn\") pod \"llmisvc-controller-manager-777c6bd7d5-ghsxn\" (UID: \"9dffc73f-6771-46be-a7ba-ab6b22d45cd5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.536116 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.536022 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:36.682846 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.682822 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn"] Jul 09 15:37:36.685487 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:37:36.685446 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9dffc73f_6771_46be_a7ba_ab6b22d45cd5.slice/crio-19bc91216889c0672418dc034aa889165fe2e9e3878bfa2a030444e0c7200e8e WatchSource:0}: Error finding container 19bc91216889c0672418dc034aa889165fe2e9e3878bfa2a030444e0c7200e8e: Status 404 returned error can't find the container with id 19bc91216889c0672418dc034aa889165fe2e9e3878bfa2a030444e0c7200e8e Jul 09 15:37:36.844827 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.844748 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" event={"ID":"9dffc73f-6771-46be-a7ba-ab6b22d45cd5","Type":"ContainerStarted","Data":"19bc91216889c0672418dc034aa889165fe2e9e3878bfa2a030444e0c7200e8e"} Jul 09 15:37:36.863379 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.863352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbc21ba3-7820-4cb8-b921-cb9f8072497a-cert\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:36.865709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:36.865693 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbc21ba3-7820-4cb8-b921-cb9f8072497a-cert\") pod \"kserve-controller-manager-d6f4b65b8-xwc9s\" (UID: \"cbc21ba3-7820-4cb8-b921-cb9f8072497a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:37.093044 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:37.093010 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:37.240941 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:37.240912 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s"] Jul 09 15:37:37.250060 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:37:37.250037 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc21ba3_7820_4cb8_b921_cb9f8072497a.slice/crio-a1cd835c37a81dffa6bba45a488e679f9967a63d978fbf3d2a1f25b19bad46e4 WatchSource:0}: Error finding container a1cd835c37a81dffa6bba45a488e679f9967a63d978fbf3d2a1f25b19bad46e4: Status 404 returned error can't find the container with id a1cd835c37a81dffa6bba45a488e679f9967a63d978fbf3d2a1f25b19bad46e4 Jul 09 15:37:37.854112 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:37.854048 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" event={"ID":"cbc21ba3-7820-4cb8-b921-cb9f8072497a","Type":"ContainerStarted","Data":"a1cd835c37a81dffa6bba45a488e679f9967a63d978fbf3d2a1f25b19bad46e4"} Jul 09 15:37:38.495019 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.494982 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv"] Jul 09 15:37:38.495525 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.495223 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" podUID="dc5c088b-4e30-4ff6-9480-f8ca6b10c506" containerName="discovery" containerID="cri-o://d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39" gracePeriod=30 Jul 09 15:37:38.752717 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.752623 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:38.788934 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.788894 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-kubeconfig\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.789117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.788957 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-local-certs\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.789117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.788994 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-cacerts\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.789117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.789052 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-ca-configmap\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.789117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.789108 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kw4d\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-kube-api-access-7kw4d\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.789345 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.789137 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-token\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.789345 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.789169 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-dns-cert\") pod \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\" (UID: \"dc5c088b-4e30-4ff6-9480-f8ca6b10c506\") " Jul 09 15:37:38.790058 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.789802 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:37:38.792480 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.792222 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:37:38.792480 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.792316 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-local-certs" (OuterVolumeSpecName: "local-certs") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:37:38.792480 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.792317 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:37:38.792480 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.792372 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:37:38.792789 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.792564 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-cacerts" (OuterVolumeSpecName: "cacerts") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:37:38.792789 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.792703 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-kube-api-access-7kw4d" (OuterVolumeSpecName: "kube-api-access-7kw4d") pod "dc5c088b-4e30-4ff6-9480-f8ca6b10c506" (UID: "dc5c088b-4e30-4ff6-9480-f8ca6b10c506"). InnerVolumeSpecName "kube-api-access-7kw4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:37:38.869146 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.869106 2577 generic.go:356] "Generic (PLEG): container finished" podID="dc5c088b-4e30-4ff6-9480-f8ca6b10c506" containerID="d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39" exitCode=0 Jul 09 15:37:38.869351 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.869173 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" Jul 09 15:37:38.869351 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.869193 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" event={"ID":"dc5c088b-4e30-4ff6-9480-f8ca6b10c506","Type":"ContainerDied","Data":"d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39"} Jul 09 15:37:38.869351 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.869259 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv" event={"ID":"dc5c088b-4e30-4ff6-9480-f8ca6b10c506","Type":"ContainerDied","Data":"a379a0f6351debb7431d791b4dd1c264e3c00f538f2d9102cc465a86b5f168bb"} Jul 09 15:37:38.869351 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.869288 2577 scope.go:117] "RemoveContainer" containerID="d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39" Jul 09 15:37:38.879388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.879367 2577 scope.go:117] "RemoveContainer" containerID="d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39" Jul 09 15:37:38.879675 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:37:38.879655 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39\": container with ID starting with d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39 not found: ID does not exist" containerID="d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39" Jul 09 15:37:38.879730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.879685 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39"} err="failed to get container status \"d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39\": rpc error: code = NotFound desc = could not find container \"d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39\": container with ID starting with d5a6736f0f19ed9dc175f1fa56a99204b38e18509f755bf96b9e26328e43aa39 not found: ID does not exist" Jul 09 15:37:38.890247 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890217 2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-kubeconfig\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.890247 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890243 2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-local-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.890397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890255 2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-cacerts\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.890397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890267 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-ca-configmap\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.890397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890278 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kw4d\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-kube-api-access-7kw4d\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.890397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890290 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.890397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.890301 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc5c088b-4e30-4ff6-9480-f8ca6b10c506-istio-csr-dns-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:37:38.898207 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.898180 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv"] Jul 09 15:37:38.899849 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:38.899827 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zw7xv"] Jul 09 15:37:40.136431 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.136400 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc5c088b-4e30-4ff6-9480-f8ca6b10c506" path="/var/lib/kubelet/pods/dc5c088b-4e30-4ff6-9480-f8ca6b10c506/volumes" Jul 09 15:37:40.888591 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.888555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" event={"ID":"cbc21ba3-7820-4cb8-b921-cb9f8072497a","Type":"ContainerStarted","Data":"80aad0cda6b13f9071be00febac7742e512e84477fe60f4ca0e3bb0515fcd49f"} Jul 09 15:37:40.888802 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.888643 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:37:40.899025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.892303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" event={"ID":"9dffc73f-6771-46be-a7ba-ab6b22d45cd5","Type":"ContainerStarted","Data":"445ef95b4b3c723bf8a3929bf4ea0a90148251b1f442399e50dd23b6602aad75"} Jul 09 15:37:40.899025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.892879 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:37:40.914188 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.914085 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" podStartSLOduration=2.341396344 podStartE2EDuration="4.914065561s" totalImagesPullingTime="2.572669217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:40.913524194 +0000 UTC m=+377.450751360" watchObservedRunningTime="2026-07-09 15:37:40.914065561 +0000 UTC m=+377.451292737" Jul 09 15:37:40.936006 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:40.935950 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" podStartSLOduration=0.869088518 podStartE2EDuration="4.935932135s" totalImagesPullingTime="4.066843617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:40.933047796 +0000 UTC m=+377.470274970" watchObservedRunningTime="2026-07-09 15:37:40.935932135 +0000 UTC m=+377.473159312" Jul 09 15:37:41.822161 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:37:41.822136 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-r9mvq" Jul 09 15:38:11.902289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:11.902254 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xwc9s" Jul 09 15:38:12.904693 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:12.904655 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ghsxn" Jul 09 15:38:25.099187 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.099158 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bqf5s"] Jul 09 15:38:25.099575 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.099546 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc5c088b-4e30-4ff6-9480-f8ca6b10c506" containerName="discovery" Jul 09 15:38:25.099575 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.099560 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc5c088b-4e30-4ff6-9480-f8ca6b10c506" containerName="discovery" Jul 09 15:38:25.099655 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.099626 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc5c088b-4e30-4ff6-9480-f8ca6b10c506" containerName="discovery" Jul 09 15:38:25.102531 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.102511 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:25.105029 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.105008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 15:38:25.105430 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.105393 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 15:38:25.105588 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.105567 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8k6sz\"" Jul 09 15:38:25.111185 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.111163 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bqf5s"] Jul 09 15:38:25.186656 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.186620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpzxh\" (UniqueName: \"kubernetes.io/projected/a7635b89-0150-4c17-b28e-a91e6723ff66-kube-api-access-hpzxh\") pod \"kuadrant-operator-catalog-bqf5s\" (UID: \"a7635b89-0150-4c17-b28e-a91e6723ff66\") " pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:25.287391 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.287352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpzxh\" (UniqueName: \"kubernetes.io/projected/a7635b89-0150-4c17-b28e-a91e6723ff66-kube-api-access-hpzxh\") pod \"kuadrant-operator-catalog-bqf5s\" (UID: \"a7635b89-0150-4c17-b28e-a91e6723ff66\") " pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:25.297754 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.297728 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpzxh\" (UniqueName: \"kubernetes.io/projected/a7635b89-0150-4c17-b28e-a91e6723ff66-kube-api-access-hpzxh\") pod \"kuadrant-operator-catalog-bqf5s\" (UID: \"a7635b89-0150-4c17-b28e-a91e6723ff66\") " pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:25.415014 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.414977 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:25.456241 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.456200 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bqf5s"] Jul 09 15:38:25.548510 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.548485 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bqf5s"] Jul 09 15:38:25.550199 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:25.550174 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7635b89_0150_4c17_b28e_a91e6723ff66.slice/crio-79cf26eff011252fb8dc3c1e34f2a81ce93a91ceec184a417cd86adb6405b3ff WatchSource:0}: Error finding container 79cf26eff011252fb8dc3c1e34f2a81ce93a91ceec184a417cd86adb6405b3ff: Status 404 returned error can't find the container with id 79cf26eff011252fb8dc3c1e34f2a81ce93a91ceec184a417cd86adb6405b3ff Jul 09 15:38:25.670816 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.670737 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwjs4"] Jul 09 15:38:25.674375 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.674359 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:25.684216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.684189 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwjs4"] Jul 09 15:38:25.690848 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.690824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7stdb\" (UniqueName: \"kubernetes.io/projected/e1309ef6-aada-4d11-a641-8e510e7f5c50-kube-api-access-7stdb\") pod \"kuadrant-operator-catalog-dwjs4\" (UID: \"e1309ef6-aada-4d11-a641-8e510e7f5c50\") " pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:25.791714 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.791679 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7stdb\" (UniqueName: \"kubernetes.io/projected/e1309ef6-aada-4d11-a641-8e510e7f5c50-kube-api-access-7stdb\") pod \"kuadrant-operator-catalog-dwjs4\" (UID: \"e1309ef6-aada-4d11-a641-8e510e7f5c50\") " pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:25.802866 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.802837 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7stdb\" (UniqueName: \"kubernetes.io/projected/e1309ef6-aada-4d11-a641-8e510e7f5c50-kube-api-access-7stdb\") pod \"kuadrant-operator-catalog-dwjs4\" (UID: \"e1309ef6-aada-4d11-a641-8e510e7f5c50\") " pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:25.984710 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:25.984614 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:26.100079 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:26.100048 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" event={"ID":"a7635b89-0150-4c17-b28e-a91e6723ff66","Type":"ContainerStarted","Data":"79cf26eff011252fb8dc3c1e34f2a81ce93a91ceec184a417cd86adb6405b3ff"} Jul 09 15:38:26.134870 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:26.134838 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1309ef6_aada_4d11_a641_8e510e7f5c50.slice/crio-f1fdbb3cb54e59745b30d44b76d3cafe96b3b4532f592fe58727dfd104d73951 WatchSource:0}: Error finding container f1fdbb3cb54e59745b30d44b76d3cafe96b3b4532f592fe58727dfd104d73951: Status 404 returned error can't find the container with id f1fdbb3cb54e59745b30d44b76d3cafe96b3b4532f592fe58727dfd104d73951 Jul 09 15:38:26.137112 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:26.137085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwjs4"] Jul 09 15:38:27.110022 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:27.109844 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" event={"ID":"e1309ef6-aada-4d11-a641-8e510e7f5c50","Type":"ContainerStarted","Data":"f1fdbb3cb54e59745b30d44b76d3cafe96b3b4532f592fe58727dfd104d73951"} Jul 09 15:38:28.116519 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.116409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" event={"ID":"e1309ef6-aada-4d11-a641-8e510e7f5c50","Type":"ContainerStarted","Data":"f472b22ad533b5d0227526a4e50e416ec7500926450b257335847ab3a9bfcb54"} Jul 09 15:38:28.117886 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.117859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" event={"ID":"a7635b89-0150-4c17-b28e-a91e6723ff66","Type":"ContainerStarted","Data":"986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730"} Jul 09 15:38:28.118004 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.117904 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" podUID="a7635b89-0150-4c17-b28e-a91e6723ff66" containerName="registry-server" containerID="cri-o://986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730" gracePeriod=2 Jul 09 15:38:28.136557 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.136497 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" podStartSLOduration=1.581796775 podStartE2EDuration="3.136480504s" totalImagesPullingTime="1.554683729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:38:28.13335292 +0000 UTC m=+424.670580093" watchObservedRunningTime="2026-07-09 15:38:28.136480504 +0000 UTC m=+424.673707677" Jul 09 15:38:28.152908 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.152854 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" podStartSLOduration=1.016901884 podStartE2EDuration="3.152837421s" totalImagesPullingTime="2.135935537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:38:28.150499062 +0000 UTC m=+424.687726236" watchObservedRunningTime="2026-07-09 15:38:28.152837421 +0000 UTC m=+424.690064594" Jul 09 15:38:28.359262 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.359237 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:28.415639 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.415603 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpzxh\" (UniqueName: \"kubernetes.io/projected/a7635b89-0150-4c17-b28e-a91e6723ff66-kube-api-access-hpzxh\") pod \"a7635b89-0150-4c17-b28e-a91e6723ff66\" (UID: \"a7635b89-0150-4c17-b28e-a91e6723ff66\") " Jul 09 15:38:28.417752 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.417727 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7635b89-0150-4c17-b28e-a91e6723ff66-kube-api-access-hpzxh" (OuterVolumeSpecName: "kube-api-access-hpzxh") pod "a7635b89-0150-4c17-b28e-a91e6723ff66" (UID: "a7635b89-0150-4c17-b28e-a91e6723ff66"). InnerVolumeSpecName "kube-api-access-hpzxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:28.517503 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:28.517437 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpzxh\" (UniqueName: \"kubernetes.io/projected/a7635b89-0150-4c17-b28e-a91e6723ff66-kube-api-access-hpzxh\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:29.123977 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.123947 2577 generic.go:356] "Generic (PLEG): container finished" podID="a7635b89-0150-4c17-b28e-a91e6723ff66" containerID="986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730" exitCode=0 Jul 09 15:38:29.124364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.124013 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" Jul 09 15:38:29.124364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.124026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" event={"ID":"a7635b89-0150-4c17-b28e-a91e6723ff66","Type":"ContainerDied","Data":"986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730"} Jul 09 15:38:29.124364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.124098 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bqf5s" event={"ID":"a7635b89-0150-4c17-b28e-a91e6723ff66","Type":"ContainerDied","Data":"79cf26eff011252fb8dc3c1e34f2a81ce93a91ceec184a417cd86adb6405b3ff"} Jul 09 15:38:29.124364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.124120 2577 scope.go:117] "RemoveContainer" containerID="986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730" Jul 09 15:38:29.133225 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.133205 2577 scope.go:117] "RemoveContainer" containerID="986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730" Jul 09 15:38:29.133510 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:38:29.133494 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730\": container with ID starting with 986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730 not found: ID does not exist" containerID="986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730" Jul 09 15:38:29.133563 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.133518 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730"} err="failed to get container status \"986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730\": rpc error: code = NotFound desc = could not find container \"986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730\": container with ID starting with 986a384c2675cd6ba438206d445040a76d3f310ae61e98398d9ae3fb2dfde730 not found: ID does not exist" Jul 09 15:38:29.146520 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.146488 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bqf5s"] Jul 09 15:38:29.150834 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:29.150806 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bqf5s"] Jul 09 15:38:30.135009 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:30.134974 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7635b89-0150-4c17-b28e-a91e6723ff66" path="/var/lib/kubelet/pods/a7635b89-0150-4c17-b28e-a91e6723ff66/volumes" Jul 09 15:38:35.985037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:35.984997 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:35.985454 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:35.985089 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:36.007527 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:36.007497 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:36.177798 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:36.177768 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dwjs4" Jul 09 15:38:40.097234 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.097203 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk"] Jul 09 15:38:40.097738 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.097561 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7635b89-0150-4c17-b28e-a91e6723ff66" containerName="registry-server" Jul 09 15:38:40.097738 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.097572 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7635b89-0150-4c17-b28e-a91e6723ff66" containerName="registry-server" Jul 09 15:38:40.097738 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.097656 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7635b89-0150-4c17-b28e-a91e6723ff66" containerName="registry-server" Jul 09 15:38:40.101155 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.101137 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.103249 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.103231 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ww2fh\"" Jul 09 15:38:40.111387 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.111366 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk"] Jul 09 15:38:40.222917 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.222884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.222917 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.222923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.223132 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.222990 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k96l\" (UniqueName: \"kubernetes.io/projected/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-kube-api-access-6k96l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.324393 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.324352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k96l\" (UniqueName: \"kubernetes.io/projected/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-kube-api-access-6k96l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.324601 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.324420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.324601 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.324444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.324816 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.324799 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.324851 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.324829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.334215 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.334184 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k96l\" (UniqueName: \"kubernetes.io/projected/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-kube-api-access-6k96l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.410510 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.410476 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:40.550918 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.547262 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk"] Jul 09 15:38:40.710565 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.710484 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg"] Jul 09 15:38:40.713282 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.713267 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.726303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.726276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg"] Jul 09 15:38:40.828367 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.828322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.828568 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.828402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6zn\" (UniqueName: \"kubernetes.io/projected/db69eeea-a3d6-42d4-8bf5-c6374630872e-kube-api-access-5c6zn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.828568 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.828432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.929364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.929322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6zn\" (UniqueName: \"kubernetes.io/projected/db69eeea-a3d6-42d4-8bf5-c6374630872e-kube-api-access-5c6zn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.929551 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.929371 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.929551 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.929450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.929822 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.929804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.929857 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.929840 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:40.940541 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:40.940517 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6zn\" (UniqueName: \"kubernetes.io/projected/db69eeea-a3d6-42d4-8bf5-c6374630872e-kube-api-access-5c6zn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:41.022764 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.022682 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:41.098202 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.098173 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb"] Jul 09 15:38:41.101742 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.101724 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.113912 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.113889 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb"] Jul 09 15:38:41.162070 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.162034 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg"] Jul 09 15:38:41.165209 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:41.165176 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb69eeea_a3d6_42d4_8bf5_c6374630872e.slice/crio-9bdd329c0f921a97363699defd61ce497254d181e9f6245752b2928b223783d1 WatchSource:0}: Error finding container 9bdd329c0f921a97363699defd61ce497254d181e9f6245752b2928b223783d1: Status 404 returned error can't find the container with id 9bdd329c0f921a97363699defd61ce497254d181e9f6245752b2928b223783d1 Jul 09 15:38:41.179991 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.179965 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" event={"ID":"db69eeea-a3d6-42d4-8bf5-c6374630872e","Type":"ContainerStarted","Data":"9bdd329c0f921a97363699defd61ce497254d181e9f6245752b2928b223783d1"} Jul 09 15:38:41.181587 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.181562 2577 generic.go:356] "Generic (PLEG): container finished" podID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerID="0633b6fd21bbf27c8fb105e100042157b10e941370067caa4ed88650c0dfd258" exitCode=0 Jul 09 15:38:41.181667 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.181628 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" event={"ID":"83216ab3-6a0c-44ae-a3c1-8806d1be5f41","Type":"ContainerDied","Data":"0633b6fd21bbf27c8fb105e100042157b10e941370067caa4ed88650c0dfd258"} Jul 09 15:38:41.181719 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.181683 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" event={"ID":"83216ab3-6a0c-44ae-a3c1-8806d1be5f41","Type":"ContainerStarted","Data":"ba52459c1c600ca3586c13fab7b774dacb43daaaf32116fe36c50ecfb08d7d70"} Jul 09 15:38:41.233005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.232965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.233138 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.233088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwn9h\" (UniqueName: \"kubernetes.io/projected/ebf730d1-ba6b-4e63-8208-4f81502ee664-kube-api-access-nwn9h\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.233195 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.233140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.334432 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.334352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.334432 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.334416 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.334652 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.334455 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwn9h\" (UniqueName: \"kubernetes.io/projected/ebf730d1-ba6b-4e63-8208-4f81502ee664-kube-api-access-nwn9h\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.334863 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.334839 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.334908 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.334843 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.345723 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.345701 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwn9h\" (UniqueName: \"kubernetes.io/projected/ebf730d1-ba6b-4e63-8208-4f81502ee664-kube-api-access-nwn9h\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.413662 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.413618 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:41.497205 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.497176 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85"] Jul 09 15:38:41.500893 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.500865 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.511661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.511634 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85"] Jul 09 15:38:41.556338 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.556313 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb"] Jul 09 15:38:41.558367 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:41.558342 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf730d1_ba6b_4e63_8208_4f81502ee664.slice/crio-ea172935af2bdeffd1712ecd9c56a4eafaeaf9b58a53d682b6b4b78fbd966218 WatchSource:0}: Error finding container ea172935af2bdeffd1712ecd9c56a4eafaeaf9b58a53d682b6b4b78fbd966218: Status 404 returned error can't find the container with id ea172935af2bdeffd1712ecd9c56a4eafaeaf9b58a53d682b6b4b78fbd966218 Jul 09 15:38:41.637830 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.637794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.637994 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.637864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56rj6\" (UniqueName: \"kubernetes.io/projected/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-kube-api-access-56rj6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.637994 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.637961 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.739107 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.739069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.739280 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.739237 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56rj6\" (UniqueName: \"kubernetes.io/projected/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-kube-api-access-56rj6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.739327 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.739304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.739451 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.739434 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.739628 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.739612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.749297 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.749274 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56rj6\" (UniqueName: \"kubernetes.io/projected/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-kube-api-access-56rj6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.813738 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.813706 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:41.952897 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:41.952857 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85"] Jul 09 15:38:41.958525 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:41.958498 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ba2662_090a_41a1_a2d6_a79ff5d2cf32.slice/crio-835f8ee503b9af526a3fe7f2b8ec214c00e49121147da569b8f927dc357f7336 WatchSource:0}: Error finding container 835f8ee503b9af526a3fe7f2b8ec214c00e49121147da569b8f927dc357f7336: Status 404 returned error can't find the container with id 835f8ee503b9af526a3fe7f2b8ec214c00e49121147da569b8f927dc357f7336 Jul 09 15:38:42.189074 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.189040 2577 generic.go:356] "Generic (PLEG): container finished" podID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerID="9b148ea90cb59ae6d9716259ea9d5affb70dff55a64b6fd6f06c2e2b6a55849a" exitCode=0 Jul 09 15:38:42.189514 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.189120 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" event={"ID":"79ba2662-090a-41a1-a2d6-a79ff5d2cf32","Type":"ContainerDied","Data":"9b148ea90cb59ae6d9716259ea9d5affb70dff55a64b6fd6f06c2e2b6a55849a"} Jul 09 15:38:42.189514 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.189181 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" event={"ID":"79ba2662-090a-41a1-a2d6-a79ff5d2cf32","Type":"ContainerStarted","Data":"835f8ee503b9af526a3fe7f2b8ec214c00e49121147da569b8f927dc357f7336"} Jul 09 15:38:42.190742 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.190717 2577 generic.go:356] "Generic (PLEG): container finished" podID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerID="5b7f4c4a047ebb3a0433d891c8832480abe4c6a7ea5a3c3f1a878ccf2fe5bb0d" exitCode=0 Jul 09 15:38:42.190863 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.190744 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" event={"ID":"ebf730d1-ba6b-4e63-8208-4f81502ee664","Type":"ContainerDied","Data":"5b7f4c4a047ebb3a0433d891c8832480abe4c6a7ea5a3c3f1a878ccf2fe5bb0d"} Jul 09 15:38:42.190863 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.190785 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" event={"ID":"ebf730d1-ba6b-4e63-8208-4f81502ee664","Type":"ContainerStarted","Data":"ea172935af2bdeffd1712ecd9c56a4eafaeaf9b58a53d682b6b4b78fbd966218"} Jul 09 15:38:42.192615 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.192588 2577 generic.go:356] "Generic (PLEG): container finished" podID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerID="56aba1ce7a22aaff6b95d5bcb293b6286879e49051ca78ac5b03486e25d6d210" exitCode=0 Jul 09 15:38:42.192731 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:42.192624 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" event={"ID":"db69eeea-a3d6-42d4-8bf5-c6374630872e","Type":"ContainerDied","Data":"56aba1ce7a22aaff6b95d5bcb293b6286879e49051ca78ac5b03486e25d6d210"} Jul 09 15:38:43.199720 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.199684 2577 generic.go:356] "Generic (PLEG): container finished" podID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerID="0d17a23783e7846d50e222cc8cf8b8132c29128ea9c372b872c8411699784ab9" exitCode=0 Jul 09 15:38:43.200221 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.199781 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" event={"ID":"db69eeea-a3d6-42d4-8bf5-c6374630872e","Type":"ContainerDied","Data":"0d17a23783e7846d50e222cc8cf8b8132c29128ea9c372b872c8411699784ab9"} Jul 09 15:38:43.201813 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.201791 2577 generic.go:356] "Generic (PLEG): container finished" podID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerID="a8fcce9dc2e845b72c343897a3a16ce4a04ca302b2a360c05f40b2307ca4f743" exitCode=0 Jul 09 15:38:43.201889 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.201859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" event={"ID":"83216ab3-6a0c-44ae-a3c1-8806d1be5f41","Type":"ContainerDied","Data":"a8fcce9dc2e845b72c343897a3a16ce4a04ca302b2a360c05f40b2307ca4f743"} Jul 09 15:38:43.203799 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.203778 2577 generic.go:356] "Generic (PLEG): container finished" podID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerID="cd15519776f39d233934121c19b04d11b03b06ac29966e6c8c745cbbae7aca38" exitCode=0 Jul 09 15:38:43.203915 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.203860 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" event={"ID":"79ba2662-090a-41a1-a2d6-a79ff5d2cf32","Type":"ContainerDied","Data":"cd15519776f39d233934121c19b04d11b03b06ac29966e6c8c745cbbae7aca38"} Jul 09 15:38:43.205966 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.205943 2577 generic.go:356] "Generic (PLEG): container finished" podID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerID="f0e7ae6fb32800b6df7f7888769fbe783b4ab1e0324fdbdbe0c53d2adf2048ec" exitCode=0 Jul 09 15:38:43.206059 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:43.206016 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" event={"ID":"ebf730d1-ba6b-4e63-8208-4f81502ee664","Type":"ContainerDied","Data":"f0e7ae6fb32800b6df7f7888769fbe783b4ab1e0324fdbdbe0c53d2adf2048ec"} Jul 09 15:38:44.213747 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.213715 2577 generic.go:356] "Generic (PLEG): container finished" podID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerID="8cf21b2438440d06f0aefae7f160460482dff280dad949928013a2d2e92ef4cf" exitCode=0 Jul 09 15:38:44.214202 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.213809 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" event={"ID":"db69eeea-a3d6-42d4-8bf5-c6374630872e","Type":"ContainerDied","Data":"8cf21b2438440d06f0aefae7f160460482dff280dad949928013a2d2e92ef4cf"} Jul 09 15:38:44.215856 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.215832 2577 generic.go:356] "Generic (PLEG): container finished" podID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerID="07f98acb2ebb4eb333ec1f0965e5ee3fe5e330720e15187ed62e10014e320023" exitCode=0 Jul 09 15:38:44.215960 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.215872 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" event={"ID":"83216ab3-6a0c-44ae-a3c1-8806d1be5f41","Type":"ContainerDied","Data":"07f98acb2ebb4eb333ec1f0965e5ee3fe5e330720e15187ed62e10014e320023"} Jul 09 15:38:44.218095 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.218069 2577 generic.go:356] "Generic (PLEG): container finished" podID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerID="3f4214fc9de534c4b0ffdc543b4eacb15c20317677e45cb4dd64336b0dcbb2ae" exitCode=0 Jul 09 15:38:44.218314 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.218143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" event={"ID":"79ba2662-090a-41a1-a2d6-a79ff5d2cf32","Type":"ContainerDied","Data":"3f4214fc9de534c4b0ffdc543b4eacb15c20317677e45cb4dd64336b0dcbb2ae"} Jul 09 15:38:44.219986 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.219969 2577 generic.go:356] "Generic (PLEG): container finished" podID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerID="4483c20d3bea7630d9a737efbd6be4b8b2c431e79eb989fa5c35839fdca7ce74" exitCode=0 Jul 09 15:38:44.220080 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:44.219988 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" event={"ID":"ebf730d1-ba6b-4e63-8208-4f81502ee664","Type":"ContainerDied","Data":"4483c20d3bea7630d9a737efbd6be4b8b2c431e79eb989fa5c35839fdca7ce74"} Jul 09 15:38:45.368908 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.368879 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:45.417172 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.417140 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:45.448503 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.448453 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:45.449608 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.449587 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:45.481740 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.481654 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwn9h\" (UniqueName: \"kubernetes.io/projected/ebf730d1-ba6b-4e63-8208-4f81502ee664-kube-api-access-nwn9h\") pod \"ebf730d1-ba6b-4e63-8208-4f81502ee664\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " Jul 09 15:38:45.481740 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.481705 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-bundle\") pod \"ebf730d1-ba6b-4e63-8208-4f81502ee664\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " Jul 09 15:38:45.481985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.481826 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-util\") pod \"ebf730d1-ba6b-4e63-8208-4f81502ee664\" (UID: \"ebf730d1-ba6b-4e63-8208-4f81502ee664\") " Jul 09 15:38:45.482353 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.482330 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-bundle" (OuterVolumeSpecName: "bundle") pod "ebf730d1-ba6b-4e63-8208-4f81502ee664" (UID: "ebf730d1-ba6b-4e63-8208-4f81502ee664"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.483932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.483904 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf730d1-ba6b-4e63-8208-4f81502ee664-kube-api-access-nwn9h" (OuterVolumeSpecName: "kube-api-access-nwn9h") pod "ebf730d1-ba6b-4e63-8208-4f81502ee664" (UID: "ebf730d1-ba6b-4e63-8208-4f81502ee664"). InnerVolumeSpecName "kube-api-access-nwn9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:45.487789 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.487765 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-util" (OuterVolumeSpecName: "util") pod "ebf730d1-ba6b-4e63-8208-4f81502ee664" (UID: "ebf730d1-ba6b-4e63-8208-4f81502ee664"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.582912 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.582875 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-util\") pod \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " Jul 09 15:38:45.582912 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.582920 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-util\") pod \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " Jul 09 15:38:45.583150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.582947 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-bundle\") pod \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " Jul 09 15:38:45.583150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.582989 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k96l\" (UniqueName: \"kubernetes.io/projected/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-kube-api-access-6k96l\") pod \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " Jul 09 15:38:45.583150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583027 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-util\") pod \"db69eeea-a3d6-42d4-8bf5-c6374630872e\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " Jul 09 15:38:45.583150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583070 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-bundle\") pod \"db69eeea-a3d6-42d4-8bf5-c6374630872e\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " Jul 09 15:38:45.583150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583092 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c6zn\" (UniqueName: \"kubernetes.io/projected/db69eeea-a3d6-42d4-8bf5-c6374630872e-kube-api-access-5c6zn\") pod \"db69eeea-a3d6-42d4-8bf5-c6374630872e\" (UID: \"db69eeea-a3d6-42d4-8bf5-c6374630872e\") " Jul 09 15:38:45.583150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583119 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56rj6\" (UniqueName: \"kubernetes.io/projected/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-kube-api-access-56rj6\") pod \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\" (UID: \"79ba2662-090a-41a1-a2d6-a79ff5d2cf32\") " Jul 09 15:38:45.583505 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583190 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-bundle\") pod \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\" (UID: \"83216ab3-6a0c-44ae-a3c1-8806d1be5f41\") " Jul 09 15:38:45.583505 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583498 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.583613 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583518 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwn9h\" (UniqueName: \"kubernetes.io/projected/ebf730d1-ba6b-4e63-8208-4f81502ee664-kube-api-access-nwn9h\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.583613 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583533 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebf730d1-ba6b-4e63-8208-4f81502ee664-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.583613 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583532 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-bundle" (OuterVolumeSpecName: "bundle") pod "79ba2662-090a-41a1-a2d6-a79ff5d2cf32" (UID: "79ba2662-090a-41a1-a2d6-a79ff5d2cf32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.583999 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.583951 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-bundle" (OuterVolumeSpecName: "bundle") pod "db69eeea-a3d6-42d4-8bf5-c6374630872e" (UID: "db69eeea-a3d6-42d4-8bf5-c6374630872e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.584090 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.584073 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-bundle" (OuterVolumeSpecName: "bundle") pod "83216ab3-6a0c-44ae-a3c1-8806d1be5f41" (UID: "83216ab3-6a0c-44ae-a3c1-8806d1be5f41"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.585683 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.585658 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-kube-api-access-56rj6" (OuterVolumeSpecName: "kube-api-access-56rj6") pod "79ba2662-090a-41a1-a2d6-a79ff5d2cf32" (UID: "79ba2662-090a-41a1-a2d6-a79ff5d2cf32"). InnerVolumeSpecName "kube-api-access-56rj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:45.585773 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.585731 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-kube-api-access-6k96l" (OuterVolumeSpecName: "kube-api-access-6k96l") pod "83216ab3-6a0c-44ae-a3c1-8806d1be5f41" (UID: "83216ab3-6a0c-44ae-a3c1-8806d1be5f41"). InnerVolumeSpecName "kube-api-access-6k96l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:45.586158 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.586131 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db69eeea-a3d6-42d4-8bf5-c6374630872e-kube-api-access-5c6zn" (OuterVolumeSpecName: "kube-api-access-5c6zn") pod "db69eeea-a3d6-42d4-8bf5-c6374630872e" (UID: "db69eeea-a3d6-42d4-8bf5-c6374630872e"). InnerVolumeSpecName "kube-api-access-5c6zn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:38:45.589133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.589111 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-util" (OuterVolumeSpecName: "util") pod "79ba2662-090a-41a1-a2d6-a79ff5d2cf32" (UID: "79ba2662-090a-41a1-a2d6-a79ff5d2cf32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.589303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.589283 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-util" (OuterVolumeSpecName: "util") pod "83216ab3-6a0c-44ae-a3c1-8806d1be5f41" (UID: "83216ab3-6a0c-44ae-a3c1-8806d1be5f41"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.590099 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.590075 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-util" (OuterVolumeSpecName: "util") pod "db69eeea-a3d6-42d4-8bf5-c6374630872e" (UID: "db69eeea-a3d6-42d4-8bf5-c6374630872e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:38:45.684216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684171 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684205 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684215 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684216 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684224 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k96l\" (UniqueName: \"kubernetes.io/projected/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-kube-api-access-6k96l\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684235 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-util\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684243 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db69eeea-a3d6-42d4-8bf5-c6374630872e-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684251 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c6zn\" (UniqueName: \"kubernetes.io/projected/db69eeea-a3d6-42d4-8bf5-c6374630872e-kube-api-access-5c6zn\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684259 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56rj6\" (UniqueName: \"kubernetes.io/projected/79ba2662-090a-41a1-a2d6-a79ff5d2cf32-kube-api-access-56rj6\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:45.684526 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:45.684268 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83216ab3-6a0c-44ae-a3c1-8806d1be5f41-bundle\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:38:46.232585 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.232552 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" Jul 09 15:38:46.232733 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.232552 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg" event={"ID":"db69eeea-a3d6-42d4-8bf5-c6374630872e","Type":"ContainerDied","Data":"9bdd329c0f921a97363699defd61ce497254d181e9f6245752b2928b223783d1"} Jul 09 15:38:46.232733 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.232677 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bdd329c0f921a97363699defd61ce497254d181e9f6245752b2928b223783d1" Jul 09 15:38:46.234534 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.234486 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" event={"ID":"83216ab3-6a0c-44ae-a3c1-8806d1be5f41","Type":"ContainerDied","Data":"ba52459c1c600ca3586c13fab7b774dacb43daaaf32116fe36c50ecfb08d7d70"} Jul 09 15:38:46.234676 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.234541 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba52459c1c600ca3586c13fab7b774dacb43daaaf32116fe36c50ecfb08d7d70" Jul 09 15:38:46.234676 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.234561 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk" Jul 09 15:38:46.236667 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.236646 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" event={"ID":"79ba2662-090a-41a1-a2d6-a79ff5d2cf32","Type":"ContainerDied","Data":"835f8ee503b9af526a3fe7f2b8ec214c00e49121147da569b8f927dc357f7336"} Jul 09 15:38:46.236771 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.236676 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85" Jul 09 15:38:46.236828 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.236680 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="835f8ee503b9af526a3fe7f2b8ec214c00e49121147da569b8f927dc357f7336" Jul 09 15:38:46.238685 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.238665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" event={"ID":"ebf730d1-ba6b-4e63-8208-4f81502ee664","Type":"ContainerDied","Data":"ea172935af2bdeffd1712ecd9c56a4eafaeaf9b58a53d682b6b4b78fbd966218"} Jul 09 15:38:46.238810 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.238689 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb" Jul 09 15:38:46.238810 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:46.238696 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea172935af2bdeffd1712ecd9c56a4eafaeaf9b58a53d682b6b4b78fbd966218" Jul 09 15:38:53.140956 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.140918 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f"] Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141272 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="pull" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141283 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="pull" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141295 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="util" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141303 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="util" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141314 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="extract" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141320 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="extract" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141329 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="pull" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141334 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="pull" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141342 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="util" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141347 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="util" Jul 09 15:38:53.141346 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141353 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="util" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141358 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="util" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141366 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141371 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141378 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141383 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141394 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="pull" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141399 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="pull" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141406 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="pull" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141411 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="pull" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141418 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="util" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141423 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="util" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141429 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141434 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141529 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="db69eeea-a3d6-42d4-8bf5-c6374630872e" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141538 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ba2662-090a-41a1-a2d6-a79ff5d2cf32" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141544 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="83216ab3-6a0c-44ae-a3c1-8806d1be5f41" containerName="extract" Jul 09 15:38:53.141709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.141551 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf730d1-ba6b-4e63-8208-4f81502ee664" containerName="extract" Jul 09 15:38:53.144703 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.144683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.147938 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.147915 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kfp8h\"" Jul 09 15:38:53.162252 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.162212 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f"] Jul 09 15:38:53.251594 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.251551 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c61015c-b982-43b6-b438-ab193f783648-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.251780 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.251602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j92zv\" (UniqueName: \"kubernetes.io/projected/4c61015c-b982-43b6-b438-ab193f783648-kube-api-access-j92zv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.352927 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.352885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j92zv\" (UniqueName: \"kubernetes.io/projected/4c61015c-b982-43b6-b438-ab193f783648-kube-api-access-j92zv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.353148 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.353022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c61015c-b982-43b6-b438-ab193f783648-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.353392 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.353371 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c61015c-b982-43b6-b438-ab193f783648-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.368028 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.367995 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j92zv\" (UniqueName: \"kubernetes.io/projected/4c61015c-b982-43b6-b438-ab193f783648-kube-api-access-j92zv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.455337 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.455221 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:38:53.601229 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:53.601196 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f"] Jul 09 15:38:53.604907 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:53.604873 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c61015c_b982_43b6_b438_ab193f783648.slice/crio-304dc7c1bd2e9ccb3c617b42a433d4ce59d3d217877105d5f3b3e56a85fe4d1c WatchSource:0}: Error finding container 304dc7c1bd2e9ccb3c617b42a433d4ce59d3d217877105d5f3b3e56a85fe4d1c: Status 404 returned error can't find the container with id 304dc7c1bd2e9ccb3c617b42a433d4ce59d3d217877105d5f3b3e56a85fe4d1c Jul 09 15:38:54.287491 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:54.287424 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" event={"ID":"4c61015c-b982-43b6-b438-ab193f783648","Type":"ContainerStarted","Data":"304dc7c1bd2e9ccb3c617b42a433d4ce59d3d217877105d5f3b3e56a85fe4d1c"} Jul 09 15:38:55.751711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.751669 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845"] Jul 09 15:38:55.754356 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.754330 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:38:55.758101 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.758078 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fkb4k\"" Jul 09 15:38:55.764776 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.764748 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 15:38:55.782203 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.782139 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845"] Jul 09 15:38:55.878356 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.878313 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj7z5\" (UniqueName: \"kubernetes.io/projected/9a958d42-ab21-4473-891e-c48491d9abc5-kube-api-access-gj7z5\") pod \"dns-operator-controller-manager-864dd78c4c-cf845\" (UID: \"9a958d42-ab21-4473-891e-c48491d9abc5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:38:55.979947 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.979911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj7z5\" (UniqueName: \"kubernetes.io/projected/9a958d42-ab21-4473-891e-c48491d9abc5-kube-api-access-gj7z5\") pod \"dns-operator-controller-manager-864dd78c4c-cf845\" (UID: \"9a958d42-ab21-4473-891e-c48491d9abc5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:38:55.993579 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:55.993542 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj7z5\" (UniqueName: \"kubernetes.io/projected/9a958d42-ab21-4473-891e-c48491d9abc5-kube-api-access-gj7z5\") pod \"dns-operator-controller-manager-864dd78c4c-cf845\" (UID: \"9a958d42-ab21-4473-891e-c48491d9abc5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:38:56.065496 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:56.065380 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:38:56.220824 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:56.220795 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845"] Jul 09 15:38:56.223285 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:56.223250 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a958d42_ab21_4473_891e_c48491d9abc5.slice/crio-9b37ce439abecdf313f5b2b1cb959cdc08e2102b45320aa99921d17f81f9230a WatchSource:0}: Error finding container 9b37ce439abecdf313f5b2b1cb959cdc08e2102b45320aa99921d17f81f9230a: Status 404 returned error can't find the container with id 9b37ce439abecdf313f5b2b1cb959cdc08e2102b45320aa99921d17f81f9230a Jul 09 15:38:56.298429 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:56.298392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" event={"ID":"9a958d42-ab21-4473-891e-c48491d9abc5","Type":"ContainerStarted","Data":"9b37ce439abecdf313f5b2b1cb959cdc08e2102b45320aa99921d17f81f9230a"} Jul 09 15:38:59.051030 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.050984 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz"] Jul 09 15:38:59.054299 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.054274 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:38:59.056795 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.056770 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-g9b2g\"" Jul 09 15:38:59.076879 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.076840 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz"] Jul 09 15:38:59.113017 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.112977 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbp29\" (UniqueName: \"kubernetes.io/projected/62429a0e-7f38-4b9d-b492-bc4a6210d4c9-kube-api-access-qbp29\") pod \"authorino-operator-75cc8d5cb7-sb8wz\" (UID: \"62429a0e-7f38-4b9d-b492-bc4a6210d4c9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:38:59.214397 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.214363 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbp29\" (UniqueName: \"kubernetes.io/projected/62429a0e-7f38-4b9d-b492-bc4a6210d4c9-kube-api-access-qbp29\") pod \"authorino-operator-75cc8d5cb7-sb8wz\" (UID: \"62429a0e-7f38-4b9d-b492-bc4a6210d4c9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:38:59.226658 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.226615 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbp29\" (UniqueName: \"kubernetes.io/projected/62429a0e-7f38-4b9d-b492-bc4a6210d4c9-kube-api-access-qbp29\") pod \"authorino-operator-75cc8d5cb7-sb8wz\" (UID: \"62429a0e-7f38-4b9d-b492-bc4a6210d4c9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:38:59.367265 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.367178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:38:59.851170 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:38:59.850997 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz"] Jul 09 15:38:59.854000 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:38:59.853971 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62429a0e_7f38_4b9d_b492_bc4a6210d4c9.slice/crio-74d7674e03332e61192288e9be0d173d3405b1633842b9bbdd7eed21d9d46a04 WatchSource:0}: Error finding container 74d7674e03332e61192288e9be0d173d3405b1633842b9bbdd7eed21d9d46a04: Status 404 returned error can't find the container with id 74d7674e03332e61192288e9be0d173d3405b1633842b9bbdd7eed21d9d46a04 Jul 09 15:39:00.328787 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:00.328746 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" event={"ID":"9a958d42-ab21-4473-891e-c48491d9abc5","Type":"ContainerStarted","Data":"23e8aadc67596e93ff5e960097c83a81e1e679093681d772e1c0eecd6917e4bc"} Jul 09 15:39:00.329229 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:00.328812 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:39:00.330556 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:00.330523 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" event={"ID":"4c61015c-b982-43b6-b438-ab193f783648","Type":"ContainerStarted","Data":"7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af"} Jul 09 15:39:00.330703 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:00.330621 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:00.331919 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:00.331896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" event={"ID":"62429a0e-7f38-4b9d-b492-bc4a6210d4c9","Type":"ContainerStarted","Data":"74d7674e03332e61192288e9be0d173d3405b1633842b9bbdd7eed21d9d46a04"} Jul 09 15:39:00.349691 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:00.349633 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" podStartSLOduration=1.879603771 podStartE2EDuration="5.349611522s" totalImagesPullingTime="3.470007751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:00.346789682 +0000 UTC m=+456.884016856" watchObservedRunningTime="2026-07-09 15:39:00.349611522 +0000 UTC m=+456.886838695" Jul 09 15:39:01.658364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.658305 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" podStartSLOduration=2.556434324 podStartE2EDuration="8.658285614s" totalImagesPullingTime="6.10185129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:00.372080903 +0000 UTC m=+456.909308078" watchObservedRunningTime="2026-07-09 15:39:01.658285614 +0000 UTC m=+458.195512787" Jul 09 15:39:01.659158 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.659134 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc"] Jul 09 15:39:01.662967 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.662942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:01.665190 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.665166 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gmchx\"" Jul 09 15:39:01.672886 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.672853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc"] Jul 09 15:39:01.739729 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.739672 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk62x\" (UniqueName: \"kubernetes.io/projected/db1612d9-2db2-4204-be48-9e94116491b5-kube-api-access-vk62x\") pod \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" (UID: \"db1612d9-2db2-4204-be48-9e94116491b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:01.841099 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.841058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk62x\" (UniqueName: \"kubernetes.io/projected/db1612d9-2db2-4204-be48-9e94116491b5-kube-api-access-vk62x\") pod \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" (UID: \"db1612d9-2db2-4204-be48-9e94116491b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:01.853195 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.853135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk62x\" (UniqueName: \"kubernetes.io/projected/db1612d9-2db2-4204-be48-9e94116491b5-kube-api-access-vk62x\") pod \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" (UID: \"db1612d9-2db2-4204-be48-9e94116491b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:01.976702 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:01.976615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:02.348681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:02.348523 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" event={"ID":"62429a0e-7f38-4b9d-b492-bc4a6210d4c9","Type":"ContainerStarted","Data":"af191e9d6c014fd4ed576c77c01f127cfbd69e0e3af437a15be6a3d9813198a1"} Jul 09 15:39:02.348681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:02.348595 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:39:02.351449 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:02.351424 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc"] Jul 09 15:39:02.351841 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:39:02.351821 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1612d9_2db2_4204_be48_9e94116491b5.slice/crio-d06c4243d885231a51d726fea1ed6bf14ff35a325367af81165c7e6ac5da1464 WatchSource:0}: Error finding container d06c4243d885231a51d726fea1ed6bf14ff35a325367af81165c7e6ac5da1464: Status 404 returned error can't find the container with id d06c4243d885231a51d726fea1ed6bf14ff35a325367af81165c7e6ac5da1464 Jul 09 15:39:02.377881 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:02.377827 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" podStartSLOduration=0.990767984 podStartE2EDuration="3.377806436s" totalImagesPullingTime="2.387038452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:02.376348838 +0000 UTC m=+458.913576012" watchObservedRunningTime="2026-07-09 15:39:02.377806436 +0000 UTC m=+458.915033610" Jul 09 15:39:03.245120 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.245060 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k"] Jul 09 15:39:03.248483 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.248438 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.251400 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.251161 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 15:39:03.251400 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.251359 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ww2fh\"" Jul 09 15:39:03.251400 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.251392 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 15:39:03.259100 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.259077 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k"] Jul 09 15:39:03.355645 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.355603 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/98c78943-6895-4e9f-8a49-a86c70fbb27f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.355841 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.355718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98c78943-6895-4e9f-8a49-a86c70fbb27f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.355841 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.355812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4kq\" (UniqueName: \"kubernetes.io/projected/98c78943-6895-4e9f-8a49-a86c70fbb27f-kube-api-access-hp4kq\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.357207 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.357177 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" event={"ID":"db1612d9-2db2-4204-be48-9e94116491b5","Type":"ContainerStarted","Data":"d06c4243d885231a51d726fea1ed6bf14ff35a325367af81165c7e6ac5da1464"} Jul 09 15:39:03.457226 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.457186 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/98c78943-6895-4e9f-8a49-a86c70fbb27f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.457410 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.457288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98c78943-6895-4e9f-8a49-a86c70fbb27f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.457410 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.457340 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4kq\" (UniqueName: \"kubernetes.io/projected/98c78943-6895-4e9f-8a49-a86c70fbb27f-kube-api-access-hp4kq\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.458192 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.458165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98c78943-6895-4e9f-8a49-a86c70fbb27f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.460051 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.460026 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/98c78943-6895-4e9f-8a49-a86c70fbb27f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.467990 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.467949 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4kq\" (UniqueName: \"kubernetes.io/projected/98c78943-6895-4e9f-8a49-a86c70fbb27f-kube-api-access-hp4kq\") pod \"kuadrant-console-plugin-66b5774985-58t2k\" (UID: \"98c78943-6895-4e9f-8a49-a86c70fbb27f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.563088 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.563004 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" Jul 09 15:39:03.720177 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:03.720147 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k"] Jul 09 15:39:03.946972 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:39:03.946938 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98c78943_6895_4e9f_8a49_a86c70fbb27f.slice/crio-b01f996ac2bf20039f7fbf4b0cbe22b2387e402d2e61adea4d4c2890b3c48795 WatchSource:0}: Error finding container b01f996ac2bf20039f7fbf4b0cbe22b2387e402d2e61adea4d4c2890b3c48795: Status 404 returned error can't find the container with id b01f996ac2bf20039f7fbf4b0cbe22b2387e402d2e61adea4d4c2890b3c48795 Jul 09 15:39:04.364175 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:04.364073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" event={"ID":"db1612d9-2db2-4204-be48-9e94116491b5","Type":"ContainerStarted","Data":"5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0"} Jul 09 15:39:04.364175 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:04.364138 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:04.365359 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:04.365337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" event={"ID":"98c78943-6895-4e9f-8a49-a86c70fbb27f","Type":"ContainerStarted","Data":"b01f996ac2bf20039f7fbf4b0cbe22b2387e402d2e61adea4d4c2890b3c48795"} Jul 09 15:39:11.343393 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:11.343361 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:11.343974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:11.343449 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cf845" Jul 09 15:39:11.368676 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:11.368611 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" podStartSLOduration=8.736277751 podStartE2EDuration="10.368588452s" totalImagesPullingTime="1.632310701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:04.429933359 +0000 UTC m=+460.967160536" watchObservedRunningTime="2026-07-09 15:39:11.368588452 +0000 UTC m=+467.905815627" Jul 09 15:39:13.299220 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.299185 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f"] Jul 09 15:39:13.299731 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.299417 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" podUID="4c61015c-b982-43b6-b438-ab193f783648" containerName="manager" containerID="cri-o://7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af" gracePeriod=2 Jul 09 15:39:13.306602 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.306515 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f"] Jul 09 15:39:13.321228 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.320921 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc"] Jul 09 15:39:13.321228 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.321185 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" podUID="db1612d9-2db2-4204-be48-9e94116491b5" containerName="manager" containerID="cri-o://5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0" gracePeriod=2 Jul 09 15:39:13.324512 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.324484 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:13.324939 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.324890 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:13.325299 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.325275 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc"] Jul 09 15:39:13.328359 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.328320 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:13.334429 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.334395 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:13.336265 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.336228 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:13.345633 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.345523 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6"] Jul 09 15:39:13.346100 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.346077 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1612d9-2db2-4204-be48-9e94116491b5" containerName="manager" Jul 09 15:39:13.346280 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.346246 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1612d9-2db2-4204-be48-9e94116491b5" containerName="manager" Jul 09 15:39:13.346398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.346308 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c61015c-b982-43b6-b438-ab193f783648" containerName="manager" Jul 09 15:39:13.346398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.346320 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c61015c-b982-43b6-b438-ab193f783648" containerName="manager" Jul 09 15:39:13.346533 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.346436 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1612d9-2db2-4204-be48-9e94116491b5" containerName="manager" Jul 09 15:39:13.346533 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.346450 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c61015c-b982-43b6-b438-ab193f783648" containerName="manager" Jul 09 15:39:13.350137 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.350113 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.356312 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.356275 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:13.358847 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.358792 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:13.359158 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.359136 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sb8wz" Jul 09 15:39:13.378258 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.378212 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6"] Jul 09 15:39:13.380704 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.380676 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc"] Jul 09 15:39:13.384755 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.384728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:13.410424 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.410389 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc"] Jul 09 15:39:13.432261 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.432215 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:13.434112 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.433958 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:13.451849 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.451812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xwck\" (UniqueName: \"kubernetes.io/projected/5af44b95-fc4b-4c27-a223-34816adc2781-kube-api-access-4xwck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-grmv6\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.452027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.451882 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5af44b95-fc4b-4c27-a223-34816adc2781-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-grmv6\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.452027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.452008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd6xc\" (UniqueName: \"kubernetes.io/projected/425a9a1a-294c-4bd2-90ed-2753e0f5114e-kube-api-access-vd6xc\") pod \"limitador-operator-controller-manager-859d7b78bd-fhcsc\" (UID: \"425a9a1a-294c-4bd2-90ed-2753e0f5114e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:13.553245 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.553156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd6xc\" (UniqueName: \"kubernetes.io/projected/425a9a1a-294c-4bd2-90ed-2753e0f5114e-kube-api-access-vd6xc\") pod \"limitador-operator-controller-manager-859d7b78bd-fhcsc\" (UID: \"425a9a1a-294c-4bd2-90ed-2753e0f5114e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:13.553245 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.553226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xwck\" (UniqueName: \"kubernetes.io/projected/5af44b95-fc4b-4c27-a223-34816adc2781-kube-api-access-4xwck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-grmv6\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.553493 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.553280 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5af44b95-fc4b-4c27-a223-34816adc2781-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-grmv6\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.553690 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.553668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5af44b95-fc4b-4c27-a223-34816adc2781-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-grmv6\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.567311 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.567280 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd6xc\" (UniqueName: \"kubernetes.io/projected/425a9a1a-294c-4bd2-90ed-2753e0f5114e-kube-api-access-vd6xc\") pod \"limitador-operator-controller-manager-859d7b78bd-fhcsc\" (UID: \"425a9a1a-294c-4bd2-90ed-2753e0f5114e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:13.569063 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.569030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xwck\" (UniqueName: \"kubernetes.io/projected/5af44b95-fc4b-4c27-a223-34816adc2781-kube-api-access-4xwck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-grmv6\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.650434 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.650393 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m"] Jul 09 15:39:13.655591 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.655563 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.657558 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.657527 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:13.659213 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:13.659185 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:13.670202 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.670176 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m"] Jul 09 15:39:13.727082 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.727042 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:13.734512 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.734488 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:13.755787 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.755718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4bf8583-f296-4467-a964-7281696e2447-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s6m\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.755787 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.755786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fdjn\" (UniqueName: \"kubernetes.io/projected/a4bf8583-f296-4467-a964-7281696e2447-kube-api-access-7fdjn\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s6m\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.856830 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.856740 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4bf8583-f296-4467-a964-7281696e2447-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s6m\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.856830 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.856799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fdjn\" (UniqueName: \"kubernetes.io/projected/a4bf8583-f296-4467-a964-7281696e2447-kube-api-access-7fdjn\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s6m\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.857221 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.857188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4bf8583-f296-4467-a964-7281696e2447-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s6m\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.873012 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.872977 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fdjn\" (UniqueName: \"kubernetes.io/projected/a4bf8583-f296-4467-a964-7281696e2447-kube-api-access-7fdjn\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s6m\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:13.970178 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:13.970139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:14.135478 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:14.135430 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:14.138317 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:14.138267 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:27.226911 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.226879 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:27.228996 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.228947 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:27.237983 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.237953 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:27.239912 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.239790 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:27.241540 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.241508 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:27.281030 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.281001 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6"] Jul 09 15:39:27.283130 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:39:27.283097 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af44b95_fc4b_4c27_a223_34816adc2781.slice/crio-ca40f7f74947db185a4262f475e66ed2c2dca71324d48c84b269eff2ca0a88be WatchSource:0}: Error finding container ca40f7f74947db185a4262f475e66ed2c2dca71324d48c84b269eff2ca0a88be: Status 404 returned error can't find the container with id ca40f7f74947db185a4262f475e66ed2c2dca71324d48c84b269eff2ca0a88be Jul 09 15:39:27.309766 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.309743 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m"] Jul 09 15:39:27.315608 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:39:27.315583 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4bf8583_f296_4467_a964_7281696e2447.slice/crio-b10d40765ff9c63311a6846f6e1a407d61e5b0ac81d26bbc3e5a224fe302131a WatchSource:0}: Error finding container b10d40765ff9c63311a6846f6e1a407d61e5b0ac81d26bbc3e5a224fe302131a: Status 404 returned error can't find the container with id b10d40765ff9c63311a6846f6e1a407d61e5b0ac81d26bbc3e5a224fe302131a Jul 09 15:39:27.325681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.325660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc"] Jul 09 15:39:27.336898 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:39:27.336874 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425a9a1a_294c_4bd2_90ed_2753e0f5114e.slice/crio-ff931c525581696da3476d63755f7bef506de6437570c3d793f43ecf821976c2 WatchSource:0}: Error finding container ff931c525581696da3476d63755f7bef506de6437570c3d793f43ecf821976c2: Status 404 returned error can't find the container with id ff931c525581696da3476d63755f7bef506de6437570c3d793f43ecf821976c2 Jul 09 15:39:27.387974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.387948 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c61015c-b982-43b6-b438-ab193f783648-extensions-socket-volume\") pod \"4c61015c-b982-43b6-b438-ab193f783648\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " Jul 09 15:39:27.388104 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.387995 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j92zv\" (UniqueName: \"kubernetes.io/projected/4c61015c-b982-43b6-b438-ab193f783648-kube-api-access-j92zv\") pod \"4c61015c-b982-43b6-b438-ab193f783648\" (UID: \"4c61015c-b982-43b6-b438-ab193f783648\") " Jul 09 15:39:27.388104 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.388027 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk62x\" (UniqueName: \"kubernetes.io/projected/db1612d9-2db2-4204-be48-9e94116491b5-kube-api-access-vk62x\") pod \"db1612d9-2db2-4204-be48-9e94116491b5\" (UID: \"db1612d9-2db2-4204-be48-9e94116491b5\") " Jul 09 15:39:27.388482 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.388445 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c61015c-b982-43b6-b438-ab193f783648-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4c61015c-b982-43b6-b438-ab193f783648" (UID: "4c61015c-b982-43b6-b438-ab193f783648"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:27.390052 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.390021 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c61015c-b982-43b6-b438-ab193f783648-kube-api-access-j92zv" (OuterVolumeSpecName: "kube-api-access-j92zv") pod "4c61015c-b982-43b6-b438-ab193f783648" (UID: "4c61015c-b982-43b6-b438-ab193f783648"). InnerVolumeSpecName "kube-api-access-j92zv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:27.390143 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.390126 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1612d9-2db2-4204-be48-9e94116491b5-kube-api-access-vk62x" (OuterVolumeSpecName: "kube-api-access-vk62x") pod "db1612d9-2db2-4204-be48-9e94116491b5" (UID: "db1612d9-2db2-4204-be48-9e94116491b5"). InnerVolumeSpecName "kube-api-access-vk62x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:27.489377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.489336 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c61015c-b982-43b6-b438-ab193f783648-extensions-socket-volume\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:39:27.489377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.489369 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j92zv\" (UniqueName: \"kubernetes.io/projected/4c61015c-b982-43b6-b438-ab193f783648-kube-api-access-j92zv\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:39:27.489377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.489383 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk62x\" (UniqueName: \"kubernetes.io/projected/db1612d9-2db2-4204-be48-9e94116491b5-kube-api-access-vk62x\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:39:27.539487 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.539366 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" event={"ID":"98c78943-6895-4e9f-8a49-a86c70fbb27f","Type":"ContainerStarted","Data":"c7d8990b9363e695a79f2b28f2bca3897ed2474dc349abb4729c55a9d8c5fef3"} Jul 09 15:39:27.542292 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.542259 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" event={"ID":"a4bf8583-f296-4467-a964-7281696e2447","Type":"ContainerStarted","Data":"3d323bfe6f11c8c7d7906b692c6022499ae92992928e0516bef4d8c413eb4a11"} Jul 09 15:39:27.542292 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.542278 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:27.542519 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.542348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" event={"ID":"a4bf8583-f296-4467-a964-7281696e2447","Type":"ContainerStarted","Data":"b10d40765ff9c63311a6846f6e1a407d61e5b0ac81d26bbc3e5a224fe302131a"} Jul 09 15:39:27.542519 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.542380 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:27.543739 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.543717 2577 generic.go:356] "Generic (PLEG): container finished" podID="4c61015c-b982-43b6-b438-ab193f783648" containerID="7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af" exitCode=0 Jul 09 15:39:27.543843 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.543763 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:27.543907 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.543822 2577 scope.go:117] "RemoveContainer" containerID="7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af" Jul 09 15:39:27.545709 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.545686 2577 generic.go:356] "Generic (PLEG): container finished" podID="db1612d9-2db2-4204-be48-9e94116491b5" containerID="5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0" exitCode=0 Jul 09 15:39:27.545857 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.545744 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:27.547670 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.547647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" event={"ID":"425a9a1a-294c-4bd2-90ed-2753e0f5114e","Type":"ContainerStarted","Data":"14627c44ffba49e3bb87f8d52808f0dfb97b8c1174dae8c28dc8e6c07ac1514d"} Jul 09 15:39:27.547770 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.547694 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" event={"ID":"425a9a1a-294c-4bd2-90ed-2753e0f5114e","Type":"ContainerStarted","Data":"ff931c525581696da3476d63755f7bef506de6437570c3d793f43ecf821976c2"} Jul 09 15:39:27.547770 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.547741 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:27.549554 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.549529 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" event={"ID":"5af44b95-fc4b-4c27-a223-34816adc2781","Type":"ContainerStarted","Data":"52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e"} Jul 09 15:39:27.549697 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.549575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" event={"ID":"5af44b95-fc4b-4c27-a223-34816adc2781","Type":"ContainerStarted","Data":"ca40f7f74947db185a4262f475e66ed2c2dca71324d48c84b269eff2ca0a88be"} Jul 09 15:39:27.549697 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.549611 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:27.555633 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.555613 2577 scope.go:117] "RemoveContainer" containerID="7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af" Jul 09 15:39:27.555947 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.555925 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af\": container with ID starting with 7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af not found: ID does not exist" containerID="7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af" Jul 09 15:39:27.556027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.555957 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af"} err="failed to get container status \"7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af\": rpc error: code = NotFound desc = could not find container \"7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af\": container with ID starting with 7801bdd7f62523b971e538f480aa71ec3507f4b8e0702fb0cfb328b0fc9004af not found: ID does not exist" Jul 09 15:39:27.556027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.555978 2577 scope.go:117] "RemoveContainer" containerID="5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0" Jul 09 15:39:27.561748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.561711 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-58t2k" podStartSLOduration=1.376410825 podStartE2EDuration="24.561693668s" totalImagesPullingTime="23.185282843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:27.561093781 +0000 UTC m=+484.098320955" watchObservedRunningTime="2026-07-09 15:39:27.561693668 +0000 UTC m=+484.098920839" Jul 09 15:39:27.562639 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.562614 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:27.564024 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.564001 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ctrlc\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="db1612d9-2db2-4204-be48-9e94116491b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ctrlc" Jul 09 15:39:27.565310 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.565294 2577 scope.go:117] "RemoveContainer" containerID="5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0" Jul 09 15:39:27.565440 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.565416 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-flm9f\" is forbidden: User \"system:node:ip-10-0-133-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-0.ec2.internal' and this object" podUID="4c61015c-b982-43b6-b438-ab193f783648" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flm9f" Jul 09 15:39:27.565589 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:27.565569 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0\": container with ID starting with 5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0 not found: ID does not exist" containerID="5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0" Jul 09 15:39:27.565647 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.565595 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0"} err="failed to get container status \"5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0\": rpc error: code = NotFound desc = could not find container \"5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0\": container with ID starting with 5e7675b355a5744ca507eb7d022cd6f7b4e7f6401fbe8c5a3601d5e155759ec0 not found: ID does not exist" Jul 09 15:39:27.593566 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.593521 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" podStartSLOduration=14.593507183 podStartE2EDuration="14.593507183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:27.591145458 +0000 UTC m=+484.128372647" watchObservedRunningTime="2026-07-09 15:39:27.593507183 +0000 UTC m=+484.130734357" Jul 09 15:39:27.617436 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.617387 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" podStartSLOduration=14.61737263 podStartE2EDuration="14.61737263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:27.616050396 +0000 UTC m=+484.153277572" watchObservedRunningTime="2026-07-09 15:39:27.61737263 +0000 UTC m=+484.154599804" Jul 09 15:39:27.636856 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:27.636802 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" podStartSLOduration=14.636787805 podStartE2EDuration="14.636787805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:27.635510913 +0000 UTC m=+484.172738088" watchObservedRunningTime="2026-07-09 15:39:27.636787805 +0000 UTC m=+484.174014980" Jul 09 15:39:28.137077 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:28.137043 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c61015c-b982-43b6-b438-ab193f783648" path="/var/lib/kubelet/pods/4c61015c-b982-43b6-b438-ab193f783648/volumes" Jul 09 15:39:28.137493 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:28.137442 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db1612d9-2db2-4204-be48-9e94116491b5" path="/var/lib/kubelet/pods/db1612d9-2db2-4204-be48-9e94116491b5/volumes" Jul 09 15:39:38.560215 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:38.560172 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:38.560692 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:38.560502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:39:38.560692 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:38.560606 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fhcsc" Jul 09 15:39:38.685845 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:38.685806 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6"] Jul 09 15:39:38.686084 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:38.686039 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" podUID="5af44b95-fc4b-4c27-a223-34816adc2781" containerName="manager" containerID="cri-o://52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e" gracePeriod=10 Jul 09 15:39:38.944697 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:38.944660 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:39.099629 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.099592 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5af44b95-fc4b-4c27-a223-34816adc2781-extensions-socket-volume\") pod \"5af44b95-fc4b-4c27-a223-34816adc2781\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " Jul 09 15:39:39.099820 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.099717 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xwck\" (UniqueName: \"kubernetes.io/projected/5af44b95-fc4b-4c27-a223-34816adc2781-kube-api-access-4xwck\") pod \"5af44b95-fc4b-4c27-a223-34816adc2781\" (UID: \"5af44b95-fc4b-4c27-a223-34816adc2781\") " Jul 09 15:39:39.100033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.100009 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5af44b95-fc4b-4c27-a223-34816adc2781-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5af44b95-fc4b-4c27-a223-34816adc2781" (UID: "5af44b95-fc4b-4c27-a223-34816adc2781"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:39:39.101932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.101898 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5af44b95-fc4b-4c27-a223-34816adc2781-kube-api-access-4xwck" (OuterVolumeSpecName: "kube-api-access-4xwck") pod "5af44b95-fc4b-4c27-a223-34816adc2781" (UID: "5af44b95-fc4b-4c27-a223-34816adc2781"). InnerVolumeSpecName "kube-api-access-4xwck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:39:39.200610 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.200568 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xwck\" (UniqueName: \"kubernetes.io/projected/5af44b95-fc4b-4c27-a223-34816adc2781-kube-api-access-4xwck\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:39:39.200610 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.200601 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5af44b95-fc4b-4c27-a223-34816adc2781-extensions-socket-volume\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:39:39.630826 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.630796 2577 generic.go:356] "Generic (PLEG): container finished" podID="5af44b95-fc4b-4c27-a223-34816adc2781" containerID="52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e" exitCode=0 Jul 09 15:39:39.631256 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.630844 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" Jul 09 15:39:39.631256 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.630858 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" event={"ID":"5af44b95-fc4b-4c27-a223-34816adc2781","Type":"ContainerDied","Data":"52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e"} Jul 09 15:39:39.631256 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.630901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6" event={"ID":"5af44b95-fc4b-4c27-a223-34816adc2781","Type":"ContainerDied","Data":"ca40f7f74947db185a4262f475e66ed2c2dca71324d48c84b269eff2ca0a88be"} Jul 09 15:39:39.631256 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.630922 2577 scope.go:117] "RemoveContainer" containerID="52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e" Jul 09 15:39:39.640439 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.640417 2577 scope.go:117] "RemoveContainer" containerID="52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e" Jul 09 15:39:39.640767 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:39:39.640743 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e\": container with ID starting with 52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e not found: ID does not exist" containerID="52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e" Jul 09 15:39:39.640844 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.640780 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e"} err="failed to get container status \"52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e\": rpc error: code = NotFound desc = could not find container \"52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e\": container with ID starting with 52fecf9c1f43123ca27e9206cbf241176217e5420c14fc973fba99af29f4c93e not found: ID does not exist" Jul 09 15:39:39.656315 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.656279 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6"] Jul 09 15:39:39.665198 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:39.665159 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-grmv6"] Jul 09 15:39:40.138449 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:40.138415 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5af44b95-fc4b-4c27-a223-34816adc2781" path="/var/lib/kubelet/pods/5af44b95-fc4b-4c27-a223-34816adc2781/volumes" Jul 09 15:39:55.200637 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.200579 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg"] Jul 09 15:39:55.202481 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.202436 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5af44b95-fc4b-4c27-a223-34816adc2781" containerName="manager" Jul 09 15:39:55.202701 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.202688 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5af44b95-fc4b-4c27-a223-34816adc2781" containerName="manager" Jul 09 15:39:55.202987 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.202975 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5af44b95-fc4b-4c27-a223-34816adc2781" containerName="manager" Jul 09 15:39:55.446259 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.446220 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg"] Jul 09 15:39:55.446419 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.446335 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.448849 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.448828 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-sr2hf\"" Jul 09 15:39:55.449067 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.449031 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 15:39:55.539442 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539412 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539620 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/976f3760-fff0-44bc-821f-3d206f41251a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539620 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/976f3760-fff0-44bc-821f-3d206f41251a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539620 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539734 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539734 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539734 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539842 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/976f3760-fff0-44bc-821f-3d206f41251a-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.539842 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.539770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5pbx\" (UniqueName: \"kubernetes.io/projected/976f3760-fff0-44bc-821f-3d206f41251a-kube-api-access-m5pbx\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640340 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640379 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/976f3760-fff0-44bc-821f-3d206f41251a-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640416 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5pbx\" (UniqueName: \"kubernetes.io/projected/976f3760-fff0-44bc-821f-3d206f41251a-kube-api-access-m5pbx\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/976f3760-fff0-44bc-821f-3d206f41251a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/976f3760-fff0-44bc-821f-3d206f41251a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.640681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.640619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.641066 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.641024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.641316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.641292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.641402 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.641303 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.642170 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.641586 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.642170 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.641738 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/976f3760-fff0-44bc-821f-3d206f41251a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.646838 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.644342 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/976f3760-fff0-44bc-821f-3d206f41251a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.646838 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.646488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/976f3760-fff0-44bc-821f-3d206f41251a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.650966 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.650939 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/976f3760-fff0-44bc-821f-3d206f41251a-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.651236 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.651213 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5pbx\" (UniqueName: \"kubernetes.io/projected/976f3760-fff0-44bc-821f-3d206f41251a-kube-api-access-m5pbx\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gphgg\" (UID: \"976f3760-fff0-44bc-821f-3d206f41251a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.756795 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.756706 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:55.917082 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:55.917055 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg"] Jul 09 15:39:55.918218 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:39:55.918186 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976f3760_fff0_44bc_821f_3d206f41251a.slice/crio-991ca2f55d0ee7fae5c8a84bfc2240ab365d011db50080e0033a4c70ef79d7f8 WatchSource:0}: Error finding container 991ca2f55d0ee7fae5c8a84bfc2240ab365d011db50080e0033a4c70ef79d7f8: Status 404 returned error can't find the container with id 991ca2f55d0ee7fae5c8a84bfc2240ab365d011db50080e0033a4c70ef79d7f8 Jul 09 15:39:56.727784 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:56.727741 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" event={"ID":"976f3760-fff0-44bc-821f-3d206f41251a","Type":"ContainerStarted","Data":"991ca2f55d0ee7fae5c8a84bfc2240ab365d011db50080e0033a4c70ef79d7f8"} Jul 09 15:39:58.418196 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:58.418160 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:58.418452 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:58.418237 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:58.418452 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:58.418266 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:39:58.740978 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:58.740879 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" event={"ID":"976f3760-fff0-44bc-821f-3d206f41251a","Type":"ContainerStarted","Data":"af2b7db1c4420085ab9f8a4c3e4898327379067a99b14d0a318857becbbc8be7"} Jul 09 15:39:58.757807 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:58.757779 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:39:58.765259 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:58.765202 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" podStartSLOduration=1.267943683 podStartE2EDuration="3.765181237s" totalImagesPullingTime="2.497237554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:58.763884009 +0000 UTC m=+515.301111181" watchObservedRunningTime="2026-07-09 15:39:58.765181237 +0000 UTC m=+515.302408413" Jul 09 15:39:59.762661 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:39:59.762633 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:40:00.388677 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.388647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:40:00.391834 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.391812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.395122 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.395101 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 15:40:00.436524 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.436491 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:40:00.471748 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.471718 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:40:00.486527 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.486495 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca775a8-1680-4ddf-96c2-873210eb6ccd-config-file\") pod \"limitador-limitador-54d6d988d5-4jk65\" (UID: \"aca775a8-1680-4ddf-96c2-873210eb6ccd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.486666 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.486600 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqwj4\" (UniqueName: \"kubernetes.io/projected/aca775a8-1680-4ddf-96c2-873210eb6ccd-kube-api-access-mqwj4\") pod \"limitador-limitador-54d6d988d5-4jk65\" (UID: \"aca775a8-1680-4ddf-96c2-873210eb6ccd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.587556 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.587519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqwj4\" (UniqueName: \"kubernetes.io/projected/aca775a8-1680-4ddf-96c2-873210eb6ccd-kube-api-access-mqwj4\") pod \"limitador-limitador-54d6d988d5-4jk65\" (UID: \"aca775a8-1680-4ddf-96c2-873210eb6ccd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.587729 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.587626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca775a8-1680-4ddf-96c2-873210eb6ccd-config-file\") pod \"limitador-limitador-54d6d988d5-4jk65\" (UID: \"aca775a8-1680-4ddf-96c2-873210eb6ccd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.588211 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.588191 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aca775a8-1680-4ddf-96c2-873210eb6ccd-config-file\") pod \"limitador-limitador-54d6d988d5-4jk65\" (UID: \"aca775a8-1680-4ddf-96c2-873210eb6ccd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.599005 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.598978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqwj4\" (UniqueName: \"kubernetes.io/projected/aca775a8-1680-4ddf-96c2-873210eb6ccd-kube-api-access-mqwj4\") pod \"limitador-limitador-54d6d988d5-4jk65\" (UID: \"aca775a8-1680-4ddf-96c2-873210eb6ccd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.702400 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.702307 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:00.752083 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.752040 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:40:00.753777 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.753267 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:40:00.860150 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:00.860128 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:40:00.861187 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:40:00.861159 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca775a8_1680_4ddf_96c2_873210eb6ccd.slice/crio-07cdbdc1e1f7d40cfc14e03c990329aed368bdf49112d67105a31180bd8ee7cd WatchSource:0}: Error finding container 07cdbdc1e1f7d40cfc14e03c990329aed368bdf49112d67105a31180bd8ee7cd: Status 404 returned error can't find the container with id 07cdbdc1e1f7d40cfc14e03c990329aed368bdf49112d67105a31180bd8ee7cd Jul 09 15:40:01.758226 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:01.758192 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" event={"ID":"aca775a8-1680-4ddf-96c2-873210eb6ccd","Type":"ContainerStarted","Data":"07cdbdc1e1f7d40cfc14e03c990329aed368bdf49112d67105a31180bd8ee7cd"} Jul 09 15:40:04.778144 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:04.778108 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" event={"ID":"aca775a8-1680-4ddf-96c2-873210eb6ccd","Type":"ContainerStarted","Data":"6018a4b9b4a63186d8be29226a9452e2ba8788d0f6183cdaf7cd28d6db571350"} Jul 09 15:40:04.778535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:04.778177 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:40:04.798590 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:04.798508 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" podStartSLOduration=1.781180948 podStartE2EDuration="4.798491056s" totalImagesPullingTime="3.017310108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:40:04.796556082 +0000 UTC m=+521.333783255" watchObservedRunningTime="2026-07-09 15:40:04.798491056 +0000 UTC m=+521.335718227" Jul 09 15:40:15.784379 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:40:15.784299 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4jk65" Jul 09 15:41:18.993860 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:18.993825 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:41:23.996436 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:23.996409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:41:23.998345 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:23.998321 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:41:27.240922 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:27.240887 2577 generic.go:356] "Generic (PLEG): container finished" podID="976f3760-fff0-44bc-821f-3d206f41251a" containerID="af2b7db1c4420085ab9f8a4c3e4898327379067a99b14d0a318857becbbc8be7" exitCode=0 Jul 09 15:41:27.241345 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:27.240961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" event={"ID":"976f3760-fff0-44bc-821f-3d206f41251a","Type":"ContainerDied","Data":"af2b7db1c4420085ab9f8a4c3e4898327379067a99b14d0a318857becbbc8be7"} Jul 09 15:41:27.241345 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:27.241331 2577 scope.go:117] "RemoveContainer" containerID="af2b7db1c4420085ab9f8a4c3e4898327379067a99b14d0a318857becbbc8be7" Jul 09 15:41:27.243598 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:27.242896 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:41:27.243598 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:27.242987 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:41:27.243598 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:27.243033 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:41:28.251928 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:28.251892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" event={"ID":"976f3760-fff0-44bc-821f-3d206f41251a","Type":"ContainerStarted","Data":"71cb03e549a6e06540f8b9648ec6b9f5e8edb4a4caaf7695143a5db8ebde578b"} Jul 09 15:41:28.757628 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:28.757591 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:41:28.758879 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:28.758833 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gphgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:41:28.759012 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:28.758909 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" podUID="976f3760-fff0-44bc-821f-3d206f41251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:41:29.757332 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:29.757294 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gphgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:41:29.757768 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:29.757353 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" podUID="976f3760-fff0-44bc-821f-3d206f41251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:41:30.751880 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:30.751825 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:41:30.757766 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:30.757729 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gphgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:41:30.758159 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:30.757785 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" podUID="976f3760-fff0-44bc-821f-3d206f41251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:41:31.757535 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:31.757497 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gphgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:41:31.757716 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:31.757557 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" podUID="976f3760-fff0-44bc-821f-3d206f41251a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 15:41:32.760745 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:32.760713 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:41:32.761839 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:32.761816 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gphgg" Jul 09 15:41:38.597720 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:38.597682 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:41:44.707223 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:41:44.707128 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:42:00.005840 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.005804 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf"] Jul 09 15:42:00.008964 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.008944 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.011267 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.011244 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 15:42:00.012066 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.012040 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4b87c\"" Jul 09 15:42:00.012179 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.012093 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 15:42:00.012179 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.012171 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 15:42:00.015182 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.015159 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.015292 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.015265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.015337 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.015297 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.015337 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.015325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.015415 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.015348 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692268f1-7155-4d38-8c4b-bfd199684495-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.015477 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.015444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cv6g\" (UniqueName: \"kubernetes.io/projected/692268f1-7155-4d38-8c4b-bfd199684495-kube-api-access-6cv6g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.021878 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.021853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf"] Jul 09 15:42:00.116158 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692268f1-7155-4d38-8c4b-bfd199684495-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116158 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cv6g\" (UniqueName: \"kubernetes.io/projected/692268f1-7155-4d38-8c4b-bfd199684495-kube-api-access-6cv6g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116416 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116416 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116416 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116416 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116892 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.116892 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116874 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.117078 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.116911 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.118769 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.118746 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692268f1-7155-4d38-8c4b-bfd199684495-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.118927 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.118910 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692268f1-7155-4d38-8c4b-bfd199684495-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.126715 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.126692 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cv6g\" (UniqueName: \"kubernetes.io/projected/692268f1-7155-4d38-8c4b-bfd199684495-kube-api-access-6cv6g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92gnf\" (UID: \"692268f1-7155-4d38-8c4b-bfd199684495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.319762 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.319658 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:00.469707 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.469668 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf"] Jul 09 15:42:00.471898 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:42:00.471868 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692268f1_7155_4d38_8c4b_bfd199684495.slice/crio-e5b4221383c22f8337998da6ee2f7784f0cfbfb8d696fa3b64b0e9841610f06a WatchSource:0}: Error finding container e5b4221383c22f8337998da6ee2f7784f0cfbfb8d696fa3b64b0e9841610f06a: Status 404 returned error can't find the container with id e5b4221383c22f8337998da6ee2f7784f0cfbfb8d696fa3b64b0e9841610f06a Jul 09 15:42:00.473312 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:00.473291 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:42:01.442692 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:01.442649 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" event={"ID":"692268f1-7155-4d38-8c4b-bfd199684495","Type":"ContainerStarted","Data":"e5b4221383c22f8337998da6ee2f7784f0cfbfb8d696fa3b64b0e9841610f06a"} Jul 09 15:42:01.500794 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:01.500759 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:42:02.449788 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:02.449752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" event={"ID":"692268f1-7155-4d38-8c4b-bfd199684495","Type":"ContainerStarted","Data":"12c3be92f154f8c4653b1ecde1537843e67cc481dbf08275d952fda15d627bcd"} Jul 09 15:42:02.471133 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:02.471077 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" podStartSLOduration=2.410667369 podStartE2EDuration="3.471059559s" totalImagesPullingTime="1.06039219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:42:02.468715256 +0000 UTC m=+639.005942429" watchObservedRunningTime="2026-07-09 15:42:02.471059559 +0000 UTC m=+639.008286733" Jul 09 15:42:10.320754 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:10.320700 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:10.321275 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:10.320780 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:10.339978 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:10.339946 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:10.505281 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:10.505249 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92gnf" Jul 09 15:42:17.329039 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:17.329003 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:42:29.017020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:29.016989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:42:40.184750 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.184719 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs"] Jul 09 15:42:40.188725 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.188707 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191114 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191114 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191116 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191149 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8fzd\" (UniqueName: \"kubernetes.io/projected/44a46cf9-215e-40e6-83c7-b76d0a99fb05-kube-api-access-k8fzd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191296 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a46cf9-215e-40e6-83c7-b76d0a99fb05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.191494 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.191322 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 15:42:40.207543 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.207512 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs"] Jul 09 15:42:40.291769 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.291732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.291769 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.291766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.291781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.291798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fzd\" (UniqueName: \"kubernetes.io/projected/44a46cf9-215e-40e6-83c7-b76d0a99fb05-kube-api-access-k8fzd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.291825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292037 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.291995 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a46cf9-215e-40e6-83c7-b76d0a99fb05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.292286 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292440 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.292340 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.292440 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.292349 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.294218 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.294199 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a46cf9-215e-40e6-83c7-b76d0a99fb05-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.294684 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.294666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a46cf9-215e-40e6-83c7-b76d0a99fb05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.305245 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.305221 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fzd\" (UniqueName: \"kubernetes.io/projected/44a46cf9-215e-40e6-83c7-b76d0a99fb05-kube-api-access-k8fzd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs\" (UID: \"44a46cf9-215e-40e6-83c7-b76d0a99fb05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.498067 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.497985 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:40.636320 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.636290 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs"] Jul 09 15:42:40.637634 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:42:40.637608 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44a46cf9_215e_40e6_83c7_b76d0a99fb05.slice/crio-12551d5aeae409d028f33dd4cdf16caca8d09fbe33089b2610fd0236df5cd96c WatchSource:0}: Error finding container 12551d5aeae409d028f33dd4cdf16caca8d09fbe33089b2610fd0236df5cd96c: Status 404 returned error can't find the container with id 12551d5aeae409d028f33dd4cdf16caca8d09fbe33089b2610fd0236df5cd96c Jul 09 15:42:40.663974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:40.663941 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" event={"ID":"44a46cf9-215e-40e6-83c7-b76d0a99fb05","Type":"ContainerStarted","Data":"12551d5aeae409d028f33dd4cdf16caca8d09fbe33089b2610fd0236df5cd96c"} Jul 09 15:42:41.671476 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:41.671435 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" event={"ID":"44a46cf9-215e-40e6-83c7-b76d0a99fb05","Type":"ContainerStarted","Data":"3d6ddc3d384c0f5dbd65602a47c943f11096064d8cb3c5e4f1f86d58ec76e813"} Jul 09 15:42:41.696422 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:41.696362 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" podStartSLOduration=1.479033339 podStartE2EDuration="1.69634684s" totalImagesPullingTime="217.313501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:42:41.693905294 +0000 UTC m=+678.231132471" watchObservedRunningTime="2026-07-09 15:42:41.69634684 +0000 UTC m=+678.233574014" Jul 09 15:42:42.900958 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:42.900920 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:42:50.498564 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:50.498533 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:50.499050 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:50.498607 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:50.510902 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:50.510876 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:42:50.742845 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:42:50.742814 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs" Jul 09 15:44:29.500903 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:44:29.500869 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:44:38.500138 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:44:38.500102 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:44:50.802595 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:44:50.802517 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:44:59.299716 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:44:59.299682 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:45:00.157021 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.156988 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp"] Jul 09 15:45:00.160313 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.160296 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" Jul 09 15:45:00.163418 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.163397 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vwgmj\"" Jul 09 15:45:00.170276 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.170252 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp"] Jul 09 15:45:00.170700 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.170686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" Jul 09 15:45:00.317841 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.313809 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp"] Jul 09 15:45:00.466158 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:00.466069 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" event={"ID":"f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3","Type":"ContainerStarted","Data":"e14a599f1525bc65a34ab181615c967360ba20e98bf0e4f9facb9038c1af96c9"} Jul 09 15:45:03.485014 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:03.484976 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" event={"ID":"f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3","Type":"ContainerStarted","Data":"7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0"} Jul 09 15:45:03.505282 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:03.505231 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" podStartSLOduration=1.276999829 podStartE2EDuration="3.505215521s" totalImagesPullingTime="2.228215692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:45:03.502398604 +0000 UTC m=+820.039625775" watchObservedRunningTime="2026-07-09 15:45:03.505215521 +0000 UTC m=+820.042442694" Jul 09 15:45:10.509999 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:10.509918 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:45:36.703253 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:36.703221 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:45:48.993239 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:45:48.993201 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:46:00.011764 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:00.011720 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp"] Jul 09 15:46:00.012344 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:00.011968 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" podUID="f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3" containerName="cleanup" containerID="cri-o://7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0" gracePeriod=30 Jul 09 15:46:23.713041 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:23.713005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:46:24.046202 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:24.046118 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:46:24.046202 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:24.046118 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:46:30.189706 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.189676 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" Jul 09 15:46:30.980190 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.980153 2577 generic.go:356] "Generic (PLEG): container finished" podID="f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3" containerID="7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0" exitCode=137 Jul 09 15:46:30.980398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.980199 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" Jul 09 15:46:30.980398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.980244 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" event={"ID":"f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3","Type":"ContainerDied","Data":"7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0"} Jul 09 15:46:30.980398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.980301 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp" event={"ID":"f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3","Type":"ContainerDied","Data":"e14a599f1525bc65a34ab181615c967360ba20e98bf0e4f9facb9038c1af96c9"} Jul 09 15:46:30.980398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.980324 2577 scope.go:117] "RemoveContainer" containerID="7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0" Jul 09 15:46:30.989747 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.989727 2577 scope.go:117] "RemoveContainer" containerID="7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0" Jul 09 15:46:30.989980 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:46:30.989963 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0\": container with ID starting with 7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0 not found: ID does not exist" containerID="7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0" Jul 09 15:46:30.990031 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:30.989987 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0"} err="failed to get container status \"7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0\": rpc error: code = NotFound desc = could not find container \"7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0\": container with ID starting with 7b5673ff3b91d7b536641944ed92357e7f60a80ad8226f1d98ddd9f71100a5a0 not found: ID does not exist" Jul 09 15:46:31.002299 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:31.002275 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp"] Jul 09 15:46:31.004731 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:31.004708 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726865-ftctp"] Jul 09 15:46:32.136006 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:32.135976 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3" path="/var/lib/kubelet/pods/f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3/volumes" Jul 09 15:46:33.895657 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:33.895624 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:46:43.797518 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:43.797482 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:46:53.891503 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:46:53.891451 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:47:03.391810 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:47:03.391775 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:47:13.517185 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:47:13.517141 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:47:25.893383 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:47:25.893350 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:47:35.703077 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:47:35.703039 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:48:00.927367 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:48:00.927276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:48:11.401626 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:48:11.401574 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:48:35.415821 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:48:35.415789 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:48:46.310427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:48:46.310387 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:49:02.796833 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:49:02.796797 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:49:14.615373 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:49:14.615335 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:49:24.097236 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:49:24.097202 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:49:33.290173 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:49:33.290139 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:49:42.885839 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:49:42.885803 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:50:08.257657 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:08.257600 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m"] Jul 09 15:50:08.258104 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:08.257826 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" podUID="a4bf8583-f296-4467-a964-7281696e2447" containerName="manager" containerID="cri-o://3d323bfe6f11c8c7d7906b692c6022499ae92992928e0516bef4d8c413eb4a11" gracePeriod=10 Jul 09 15:50:11.210432 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.210402 2577 generic.go:356] "Generic (PLEG): container finished" podID="a4bf8583-f296-4467-a964-7281696e2447" containerID="3d323bfe6f11c8c7d7906b692c6022499ae92992928e0516bef4d8c413eb4a11" exitCode=0 Jul 09 15:50:11.210823 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.210446 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" event={"ID":"a4bf8583-f296-4467-a964-7281696e2447","Type":"ContainerDied","Data":"3d323bfe6f11c8c7d7906b692c6022499ae92992928e0516bef4d8c413eb4a11"} Jul 09 15:50:11.411564 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.411540 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:50:11.475654 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.475572 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fdjn\" (UniqueName: \"kubernetes.io/projected/a4bf8583-f296-4467-a964-7281696e2447-kube-api-access-7fdjn\") pod \"a4bf8583-f296-4467-a964-7281696e2447\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " Jul 09 15:50:11.475803 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.475682 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4bf8583-f296-4467-a964-7281696e2447-extensions-socket-volume\") pod \"a4bf8583-f296-4467-a964-7281696e2447\" (UID: \"a4bf8583-f296-4467-a964-7281696e2447\") " Jul 09 15:50:11.476013 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.475991 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bf8583-f296-4467-a964-7281696e2447-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a4bf8583-f296-4467-a964-7281696e2447" (UID: "a4bf8583-f296-4467-a964-7281696e2447"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:50:11.477685 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.477652 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4bf8583-f296-4467-a964-7281696e2447-kube-api-access-7fdjn" (OuterVolumeSpecName: "kube-api-access-7fdjn") pod "a4bf8583-f296-4467-a964-7281696e2447" (UID: "a4bf8583-f296-4467-a964-7281696e2447"). InnerVolumeSpecName "kube-api-access-7fdjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:50:11.577393 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.577358 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4bf8583-f296-4467-a964-7281696e2447-extensions-socket-volume\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:50:11.577393 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:11.577386 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fdjn\" (UniqueName: \"kubernetes.io/projected/a4bf8583-f296-4467-a964-7281696e2447-kube-api-access-7fdjn\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:50:12.216877 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:12.216843 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" event={"ID":"a4bf8583-f296-4467-a964-7281696e2447","Type":"ContainerDied","Data":"b10d40765ff9c63311a6846f6e1a407d61e5b0ac81d26bbc3e5a224fe302131a"} Jul 09 15:50:12.217289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:12.216928 2577 scope.go:117] "RemoveContainer" containerID="3d323bfe6f11c8c7d7906b692c6022499ae92992928e0516bef4d8c413eb4a11" Jul 09 15:50:12.217289 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:12.216858 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m" Jul 09 15:50:12.236304 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:12.236278 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m"] Jul 09 15:50:12.240844 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:12.240820 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s6m"] Jul 09 15:50:14.136501 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:50:14.136451 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4bf8583-f296-4467-a964-7281696e2447" path="/var/lib/kubelet/pods/a4bf8583-f296-4467-a964-7281696e2447/volumes" Jul 09 15:51:09.478909 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.478833 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5"] Jul 09 15:51:09.479343 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.479210 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3" containerName="cleanup" Jul 09 15:51:09.479343 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.479221 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3" containerName="cleanup" Jul 09 15:51:09.479343 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.479233 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4bf8583-f296-4467-a964-7281696e2447" containerName="manager" Jul 09 15:51:09.479343 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.479239 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4bf8583-f296-4467-a964-7281696e2447" containerName="manager" Jul 09 15:51:09.479343 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.479310 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4bf8583-f296-4467-a964-7281696e2447" containerName="manager" Jul 09 15:51:09.479343 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.479319 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c39d5b-1568-48b6-bf8f-6deb9bc7e9f3" containerName="cleanup" Jul 09 15:51:09.481881 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.481864 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.483832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.483812 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kfp8h\"" Jul 09 15:51:09.493670 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.493648 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5"] Jul 09 15:51:09.590786 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.590753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6de3174-5ce8-4b25-ac0f-cda8cb199875-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnbh5\" (UID: \"e6de3174-5ce8-4b25-ac0f-cda8cb199875\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.590983 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.590870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wcs2\" (UniqueName: \"kubernetes.io/projected/e6de3174-5ce8-4b25-ac0f-cda8cb199875-kube-api-access-2wcs2\") pod \"kuadrant-operator-controller-manager-86878c769f-vnbh5\" (UID: \"e6de3174-5ce8-4b25-ac0f-cda8cb199875\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.691836 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.691800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wcs2\" (UniqueName: \"kubernetes.io/projected/e6de3174-5ce8-4b25-ac0f-cda8cb199875-kube-api-access-2wcs2\") pod \"kuadrant-operator-controller-manager-86878c769f-vnbh5\" (UID: \"e6de3174-5ce8-4b25-ac0f-cda8cb199875\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.691997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.691867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6de3174-5ce8-4b25-ac0f-cda8cb199875-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnbh5\" (UID: \"e6de3174-5ce8-4b25-ac0f-cda8cb199875\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.692177 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.692160 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6de3174-5ce8-4b25-ac0f-cda8cb199875-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnbh5\" (UID: \"e6de3174-5ce8-4b25-ac0f-cda8cb199875\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.699940 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.699920 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wcs2\" (UniqueName: \"kubernetes.io/projected/e6de3174-5ce8-4b25-ac0f-cda8cb199875-kube-api-access-2wcs2\") pod \"kuadrant-operator-controller-manager-86878c769f-vnbh5\" (UID: \"e6de3174-5ce8-4b25-ac0f-cda8cb199875\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.792197 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.792095 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:09.926890 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:09.926858 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5"] Jul 09 15:51:09.928125 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:51:09.928102 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6de3174_5ce8_4b25_ac0f_cda8cb199875.slice/crio-7a927aebe9fd2d1d1667a0eec6314d270cc0b00ec8165c6b632a6d9f2598e474 WatchSource:0}: Error finding container 7a927aebe9fd2d1d1667a0eec6314d270cc0b00ec8165c6b632a6d9f2598e474: Status 404 returned error can't find the container with id 7a927aebe9fd2d1d1667a0eec6314d270cc0b00ec8165c6b632a6d9f2598e474 Jul 09 15:51:10.536398 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:10.536361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" event={"ID":"e6de3174-5ce8-4b25-ac0f-cda8cb199875","Type":"ContainerStarted","Data":"0e0dc177ccbd0c3362ef19d42f973ab3b1746d6baa9ad4ab37fb372ab7030e01"} Jul 09 15:51:10.536922 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:10.536416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" event={"ID":"e6de3174-5ce8-4b25-ac0f-cda8cb199875","Type":"ContainerStarted","Data":"7a927aebe9fd2d1d1667a0eec6314d270cc0b00ec8165c6b632a6d9f2598e474"} Jul 09 15:51:10.536922 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:10.536556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:10.557559 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:10.557513 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" podStartSLOduration=1.557495652 podStartE2EDuration="1.557495652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:51:10.556240873 +0000 UTC m=+1187.093468045" watchObservedRunningTime="2026-07-09 15:51:10.557495652 +0000 UTC m=+1187.094722819" Jul 09 15:51:21.544908 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:21.544879 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnbh5" Jul 09 15:51:24.082405 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:24.082376 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:51:24.083332 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:24.083312 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:51:29.998682 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:29.998648 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:51:40.589394 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:40.589356 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:51:50.583867 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:51:50.583829 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:52:02.285777 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:52:02.285742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:52:13.490442 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:52:13.490402 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:52:23.290350 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:52:23.290316 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:52:32.896587 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:52:32.896552 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:52:43.483347 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:52:43.483310 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:52:55.180438 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:52:55.180402 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:53:04.184763 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:53:04.184732 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:53:14.389814 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:53:14.389783 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:53:36.688347 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:53:36.688310 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:53:45.609666 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:53:45.609573 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:53:55.788778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:53:55.788740 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:54:06.286564 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:54:06.286523 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:54:13.685013 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:54:13.684975 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:54:25.289308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:54:25.289250 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:54:34.403091 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:54:34.403053 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:54:43.291102 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:54:43.291063 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:54:53.494675 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:54:53.494644 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:55:02.191970 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:55:02.191929 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:55:11.188603 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:55:11.188551 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:55:24.391341 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:55:24.391308 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:55:33.490111 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:55:33.490075 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:55:43.385085 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:55:43.385053 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:55:55.885402 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:55:55.885364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:56:06.485245 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:06.485207 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:56:16.504429 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:16.504389 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:56:24.117679 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:24.117655 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:56:24.120349 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:24.120320 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 15:56:29.207125 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:29.207092 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:56:41.699028 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:41.698990 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:56:54.701377 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:56:54.701297 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:57:07.114015 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:07.113982 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:57:15.694118 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:15.694084 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:57:26.019970 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.019929 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v"] Jul 09 15:57:26.023760 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.023734 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.025997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.025865 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-077a59c2-openshift-default-dockercfg-sfblp\"" Jul 09 15:57:26.042718 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.042693 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v"] Jul 09 15:57:26.085865 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.085832 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:57:26.113605 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-credential-socket\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113605 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-socket\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113826 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-envoy\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113826 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-certs\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113826 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-token\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113992 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlnc4\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-kube-api-access-zlnc4\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113992 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istiod-ca-cert\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.113992 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.113972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-data\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.114118 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.114000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-podinfo\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214399 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-certs\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214611 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214405 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-token\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214611 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlnc4\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-kube-api-access-zlnc4\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214611 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istiod-ca-cert\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214611 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-data\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214831 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-podinfo\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214831 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-credential-socket\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214831 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-socket\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214831 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-envoy\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.214831 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.214812 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-certs\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.215155 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.215137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-credential-socket\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.215212 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.215137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-data\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.215212 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.215188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-socket\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.215316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.215277 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istiod-ca-cert\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.216864 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.216840 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-envoy\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.217036 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.217021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-podinfo\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.222484 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.222450 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-token\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.222812 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.222792 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlnc4\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-kube-api-access-zlnc4\") pod \"e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.337735 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.337633 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:26.480993 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.480969 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v"] Jul 09 15:57:26.483686 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:57:26.483658 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe80e9a_0f16_4e08_8b34_00e2a21c2644.slice/crio-37d44e2eacd5c006b9083f7c7847ff113c902d449007943106217284255cafdf WatchSource:0}: Error finding container 37d44e2eacd5c006b9083f7c7847ff113c902d449007943106217284255cafdf: Status 404 returned error can't find the container with id 37d44e2eacd5c006b9083f7c7847ff113c902d449007943106217284255cafdf Jul 09 15:57:26.486276 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.486244 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:26.486381 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.486321 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:26.486381 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.486363 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:26.632113 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.632078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" event={"ID":"9fe80e9a-0f16-4e08-8b34-00e2a21c2644","Type":"ContainerStarted","Data":"6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15"} Jul 09 15:57:26.632308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.632138 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" event={"ID":"9fe80e9a-0f16-4e08-8b34-00e2a21c2644","Type":"ContainerStarted","Data":"37d44e2eacd5c006b9083f7c7847ff113c902d449007943106217284255cafdf"} Jul 09 15:57:26.654181 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:26.654128 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" podStartSLOduration=1.654112678 podStartE2EDuration="1.654112678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:57:26.652793598 +0000 UTC m=+1563.190020772" watchObservedRunningTime="2026-07-09 15:57:26.654112678 +0000 UTC m=+1563.191339850" Jul 09 15:57:27.338410 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:27.338370 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:27.343337 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:27.343310 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:27.638816 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:27.638784 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:27.639810 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:27.639785 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:32.553603 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:32.553567 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v"] Jul 09 15:57:32.554125 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:32.553794 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" podUID="9fe80e9a-0f16-4e08-8b34-00e2a21c2644" containerName="istio-proxy" containerID="cri-o://6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15" gracePeriod=30 Jul 09 15:57:39.096353 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.096331 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:39.151432 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.151360 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd"] Jul 09 15:57:39.151985 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.151969 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe80e9a-0f16-4e08-8b34-00e2a21c2644" containerName="istio-proxy" Jul 09 15:57:39.152033 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.151989 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe80e9a-0f16-4e08-8b34-00e2a21c2644" containerName="istio-proxy" Jul 09 15:57:39.152118 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.152107 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe80e9a-0f16-4e08-8b34-00e2a21c2644" containerName="istio-proxy" Jul 09 15:57:39.155435 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.155417 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.157337 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.157317 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-abf2097d-openshift-default-dockercfg-ntr86\"" Jul 09 15:57:39.165700 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.165677 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd"] Jul 09 15:57:39.234404 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234373 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istiod-ca-cert\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234554 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234453 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-socket\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234554 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234507 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlnc4\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-kube-api-access-zlnc4\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234554 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234543 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-credential-socket\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234558 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-envoy\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234572 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-data\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234633 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-certs\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234657 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-podinfo\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.234730 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234683 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-token\") pod \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\" (UID: \"9fe80e9a-0f16-4e08-8b34-00e2a21c2644\") " Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234735 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234763 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-token\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234838 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-certs\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234954 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-data\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235020 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.234991 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:39.235442 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vs2\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-kube-api-access-d8vs2\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235442 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-credential-socket\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235442 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235177 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-podinfo\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235521 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:39.235627 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-socket\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235740 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235700 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-envoy\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.235864 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.235844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istiod-ca-cert\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.236054 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.236037 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.236117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.236062 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.236117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.236077 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.236117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.236091 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.236117 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.236103 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.236941 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.236922 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:57:39.237126 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.237105 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-kube-api-access-zlnc4" (OuterVolumeSpecName: "kube-api-access-zlnc4") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "kube-api-access-zlnc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:57:39.237180 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.237150 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:39.237217 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.237205 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fe80e9a-0f16-4e08-8b34-00e2a21c2644" (UID: "9fe80e9a-0f16-4e08-8b34-00e2a21c2644"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:57:39.336966 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.336932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-credential-socket\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337129 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.336974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-podinfo\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337129 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-socket\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337129 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-envoy\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337298 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istiod-ca-cert\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337298 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-token\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-certs\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-data\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337347 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-credential-socket\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337368 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-socket\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337408 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vs2\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-kube-api-access-d8vs2\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337513 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.337711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337528 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.337711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337541 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.337711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337553 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlnc4\" (UniqueName: \"kubernetes.io/projected/9fe80e9a-0f16-4e08-8b34-00e2a21c2644-kube-api-access-zlnc4\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:39.337711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337634 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-data\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.337711 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-certs\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.338023 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.337903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istiod-ca-cert\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.339190 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.339169 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-envoy\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.339414 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.339397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-podinfo\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.345993 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.345976 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vs2\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-kube-api-access-d8vs2\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.346181 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.346166 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-token\") pod \"e2e-ait-abf2097d-openshift-default-658b598b6f-672rd\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.467406 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.467332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:39.604221 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.604199 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd"] Jul 09 15:57:39.605272 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:57:39.605248 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ab3537_ff48_4037_ad8b_4ebe926c20f5.slice/crio-240e11cec616b92f8e9284872f0e0b6d4b086249a04c2103fae717a30f8267e1 WatchSource:0}: Error finding container 240e11cec616b92f8e9284872f0e0b6d4b086249a04c2103fae717a30f8267e1: Status 404 returned error can't find the container with id 240e11cec616b92f8e9284872f0e0b6d4b086249a04c2103fae717a30f8267e1 Jul 09 15:57:39.607295 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.607267 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:39.607363 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.607350 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:39.607400 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.607381 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:39.709862 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.709808 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" event={"ID":"27ab3537-ff48-4037-ad8b-4ebe926c20f5","Type":"ContainerStarted","Data":"9edc390aa48ab1aa77bb49ba7cfdd9045d39a01053700bc9d9513a77fbfa2aa8"} Jul 09 15:57:39.710027 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.709875 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" event={"ID":"27ab3537-ff48-4037-ad8b-4ebe926c20f5","Type":"ContainerStarted","Data":"240e11cec616b92f8e9284872f0e0b6d4b086249a04c2103fae717a30f8267e1"} Jul 09 15:57:39.711204 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.711180 2577 generic.go:356] "Generic (PLEG): container finished" podID="9fe80e9a-0f16-4e08-8b34-00e2a21c2644" containerID="6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15" exitCode=0 Jul 09 15:57:39.711341 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.711248 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" Jul 09 15:57:39.711341 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.711259 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" event={"ID":"9fe80e9a-0f16-4e08-8b34-00e2a21c2644","Type":"ContainerDied","Data":"6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15"} Jul 09 15:57:39.711341 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.711315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v" event={"ID":"9fe80e9a-0f16-4e08-8b34-00e2a21c2644","Type":"ContainerDied","Data":"37d44e2eacd5c006b9083f7c7847ff113c902d449007943106217284255cafdf"} Jul 09 15:57:39.711549 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.711347 2577 scope.go:117] "RemoveContainer" containerID="6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15" Jul 09 15:57:39.720624 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.720608 2577 scope.go:117] "RemoveContainer" containerID="6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15" Jul 09 15:57:39.720957 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:57:39.720923 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15\": container with ID starting with 6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15 not found: ID does not exist" containerID="6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15" Jul 09 15:57:39.721052 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.720966 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15"} err="failed to get container status \"6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15\": rpc error: code = NotFound desc = could not find container \"6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15\": container with ID starting with 6068983f192b2fa26bad952b1674a93cd1a624e6f36dc315e2ae2897852f6c15 not found: ID does not exist" Jul 09 15:57:39.729708 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.729660 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" podStartSLOduration=0.729645272 podStartE2EDuration="729.645272ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:57:39.729530716 +0000 UTC m=+1576.266757889" watchObservedRunningTime="2026-07-09 15:57:39.729645272 +0000 UTC m=+1576.266872447" Jul 09 15:57:39.743948 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.743893 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v"] Jul 09 15:57:39.749571 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:39.749547 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-077a59c2-openshift-default-78b6c88479-jqf2v"] Jul 09 15:57:40.138378 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:40.138339 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe80e9a-0f16-4e08-8b34-00e2a21c2644" path="/var/lib/kubelet/pods/9fe80e9a-0f16-4e08-8b34-00e2a21c2644/volumes" Jul 09 15:57:40.468387 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:40.468299 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:40.480315 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:40.480289 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:40.719596 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:40.719524 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:40.720522 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:40.720497 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:51.589300 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:51.589263 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd"] Jul 09 15:57:51.589771 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:51.589505 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" podUID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" containerName="istio-proxy" containerID="cri-o://9edc390aa48ab1aa77bb49ba7cfdd9045d39a01053700bc9d9513a77fbfa2aa8" gracePeriod=30 Jul 09 15:57:55.721668 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:55.721636 2577 patch_prober.go:28] interesting pod/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 15:57:55.722056 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:55.721694 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" podUID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 15:57:56.815051 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:56.815023 2577 generic.go:356] "Generic (PLEG): container finished" podID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" containerID="9edc390aa48ab1aa77bb49ba7cfdd9045d39a01053700bc9d9513a77fbfa2aa8" exitCode=0 Jul 09 15:57:56.815388 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:56.815068 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" event={"ID":"27ab3537-ff48-4037-ad8b-4ebe926c20f5","Type":"ContainerDied","Data":"9edc390aa48ab1aa77bb49ba7cfdd9045d39a01053700bc9d9513a77fbfa2aa8"} Jul 09 15:57:56.845785 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:56.845764 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:57.000676 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000649 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-certs\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.000832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000686 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-data\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.000832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000755 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-socket\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.000832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000796 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-token\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.001001 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000848 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-envoy\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.001001 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000889 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8vs2\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-kube-api-access-d8vs2\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.001001 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000900 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:57.001001 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000943 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-podinfo\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.001001 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.000980 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istiod-ca-cert\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.001235 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001006 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-credential-socket\") pod \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\" (UID: \"27ab3537-ff48-4037-ad8b-4ebe926c20f5\") " Jul 09 15:57:57.001235 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001051 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:57.001235 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001067 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:57.001378 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001292 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:57.001378 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001367 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:57:57.001509 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001392 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.001509 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001500 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.001591 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001519 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.001591 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.001533 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.002963 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.002938 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:57:57.003079 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.003042 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:57:57.003233 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.003206 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:57:57.003325 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.003254 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-kube-api-access-d8vs2" (OuterVolumeSpecName: "kube-api-access-d8vs2") pod "27ab3537-ff48-4037-ad8b-4ebe926c20f5" (UID: "27ab3537-ff48-4037-ad8b-4ebe926c20f5"). InnerVolumeSpecName "kube-api-access-d8vs2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:57:57.102086 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.102058 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.102086 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.102083 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.102086 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.102092 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.102316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.102100 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab3537-ff48-4037-ad8b-4ebe926c20f5-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.102316 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.102109 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8vs2\" (UniqueName: \"kubernetes.io/projected/27ab3537-ff48-4037-ad8b-4ebe926c20f5-kube-api-access-d8vs2\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:57:57.821236 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.821204 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" Jul 09 15:57:57.821751 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.821247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd" event={"ID":"27ab3537-ff48-4037-ad8b-4ebe926c20f5","Type":"ContainerDied","Data":"240e11cec616b92f8e9284872f0e0b6d4b086249a04c2103fae717a30f8267e1"} Jul 09 15:57:57.821751 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.821314 2577 scope.go:117] "RemoveContainer" containerID="9edc390aa48ab1aa77bb49ba7cfdd9045d39a01053700bc9d9513a77fbfa2aa8" Jul 09 15:57:57.842997 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.842975 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd"] Jul 09 15:57:57.847512 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:57.847491 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-abf2097d-openshift-default-658b598b6f-672rd"] Jul 09 15:57:58.136804 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.136776 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" path="/var/lib/kubelet/pods/27ab3537-ff48-4037-ad8b-4ebe926c20f5/volumes" Jul 09 15:57:58.186922 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.186890 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z"] Jul 09 15:57:58.187298 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.187285 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" containerName="istio-proxy" Jul 09 15:57:58.187364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.187299 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" containerName="istio-proxy" Jul 09 15:57:58.187364 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.187360 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="27ab3537-ff48-4037-ad8b-4ebe926c20f5" containerName="istio-proxy" Jul 09 15:57:58.191788 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.191767 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.194021 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.193928 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-545edf81-openshift-default-dockercfg-5mxcd\"" Jul 09 15:57:58.201832 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.201808 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z"] Jul 09 15:57:58.313262 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-certs\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313271 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-token\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27f4q\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-kube-api-access-27f4q\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313427 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-podinfo\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313571 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-socket\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313571 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-credential-socket\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313571 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-envoy\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313571 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istiod-ca-cert\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.313743 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.313594 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-data\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415044 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.414963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-token\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415044 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.414999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27f4q\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-kube-api-access-27f4q\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415044 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-podinfo\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415091 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-socket\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-credential-socket\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-envoy\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415207 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istiod-ca-cert\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-data\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415308 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-certs\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415648 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415582 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-socket\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415648 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-credential-socket\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415742 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415722 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-certs\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415861 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-data\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.415960 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.415943 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istiod-ca-cert\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.417372 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.417346 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-envoy\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.417497 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.417484 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-podinfo\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.424741 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.424713 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-token\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.427258 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.427237 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27f4q\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-kube-api-access-27f4q\") pod \"e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.503194 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.503164 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:58.640135 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.640105 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z"] Jul 09 15:57:58.641788 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:57:58.641758 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1dba27_b2da_45bc_90db_0e6f47d36e0a.slice/crio-eb544cef0292e4c157b935a3ad8de07c2133494a827456c0d8a5c0e733ef8c18 WatchSource:0}: Error finding container eb544cef0292e4c157b935a3ad8de07c2133494a827456c0d8a5c0e733ef8c18: Status 404 returned error can't find the container with id eb544cef0292e4c157b935a3ad8de07c2133494a827456c0d8a5c0e733ef8c18 Jul 09 15:57:58.644211 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.644180 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:58.644282 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.644240 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:58.644282 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.644265 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:57:58.830672 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.830625 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" event={"ID":"ce1dba27-b2da-45bc-90db-0e6f47d36e0a","Type":"ContainerStarted","Data":"d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da"} Jul 09 15:57:58.831044 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.830686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" event={"ID":"ce1dba27-b2da-45bc-90db-0e6f47d36e0a","Type":"ContainerStarted","Data":"eb544cef0292e4c157b935a3ad8de07c2133494a827456c0d8a5c0e733ef8c18"} Jul 09 15:57:58.852330 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:58.852270 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" podStartSLOduration=0.85225081 podStartE2EDuration="852.25081ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:57:58.84887885 +0000 UTC m=+1595.386106024" watchObservedRunningTime="2026-07-09 15:57:58.85225081 +0000 UTC m=+1595.389477985" Jul 09 15:57:59.503943 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:59.503907 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:59.508974 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:59.508951 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:59.837376 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:59.837294 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:57:59.838378 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:57:59.838356 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:58:02.281993 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:02.281961 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:58:04.025292 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:04.025252 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z"] Jul 09 15:58:04.025803 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:04.025490 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" podUID="ce1dba27-b2da-45bc-90db-0e6f47d36e0a" containerName="istio-proxy" containerID="cri-o://d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da" gracePeriod=30 Jul 09 15:58:07.291435 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:07.291361 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:58:10.107274 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.107251 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:58:10.126081 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126052 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-certs\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.126224 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126123 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-data\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.126292 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126210 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27f4q\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-kube-api-access-27f4q\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.126292 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126272 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-socket\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.126395 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126310 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-envoy\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.126480 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126450 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istiod-ca-cert\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.129244 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-credential-socket\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.129284 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-token\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.129329 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-podinfo\") pod \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\" (UID: \"ce1dba27-b2da-45bc-90db-0e6f47d36e0a\") " Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126338 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.126836 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.127020 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.127033 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.128996 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:58:10.130778 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.130097 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:58:10.132206 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.132160 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:58:10.132446 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.132417 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-kube-api-access-27f4q" (OuterVolumeSpecName: "kube-api-access-27f4q") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "kube-api-access-27f4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:58:10.134144 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.133203 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce1dba27-b2da-45bc-90db-0e6f47d36e0a" (UID: "ce1dba27-b2da-45bc-90db-0e6f47d36e0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:58:10.231084 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231050 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231084 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231077 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231084 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231089 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231098 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27f4q\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-kube-api-access-27f4q\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231106 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231114 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231121 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231131 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.231303 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.231142 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1dba27-b2da-45bc-90db-0e6f47d36e0a-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 15:58:10.899746 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.899711 2577 generic.go:356] "Generic (PLEG): container finished" podID="ce1dba27-b2da-45bc-90db-0e6f47d36e0a" containerID="d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da" exitCode=0 Jul 09 15:58:10.899932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.899756 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" Jul 09 15:58:10.899932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.899799 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" event={"ID":"ce1dba27-b2da-45bc-90db-0e6f47d36e0a","Type":"ContainerDied","Data":"d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da"} Jul 09 15:58:10.899932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.899850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z" event={"ID":"ce1dba27-b2da-45bc-90db-0e6f47d36e0a","Type":"ContainerDied","Data":"eb544cef0292e4c157b935a3ad8de07c2133494a827456c0d8a5c0e733ef8c18"} Jul 09 15:58:10.899932 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.899871 2577 scope.go:117] "RemoveContainer" containerID="d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da" Jul 09 15:58:10.908824 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.908806 2577 scope.go:117] "RemoveContainer" containerID="d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da" Jul 09 15:58:10.909083 ip-10-0-133-0 kubenswrapper[2577]: E0709 15:58:10.909052 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da\": container with ID starting with d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da not found: ID does not exist" containerID="d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da" Jul 09 15:58:10.909132 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.909095 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da"} err="failed to get container status \"d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da\": rpc error: code = NotFound desc = could not find container \"d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da\": container with ID starting with d989e73d5ff751a2172fc6f60815277157c85d6b0719d99488ff12ac743571da not found: ID does not exist" Jul 09 15:58:10.917069 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.917045 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z"] Jul 09 15:58:10.921447 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:10.921426 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-545edf81-openshift-default-766bd8d8bc-ts69z"] Jul 09 15:58:12.136576 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:58:12.136543 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce1dba27-b2da-45bc-90db-0e6f47d36e0a" path="/var/lib/kubelet/pods/ce1dba27-b2da-45bc-90db-0e6f47d36e0a/volumes" Jul 09 15:59:34.186776 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:34.186747 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:59:41.588273 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:41.588234 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:59:48.102222 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.102134 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2"] Jul 09 15:59:48.102785 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.102769 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce1dba27-b2da-45bc-90db-0e6f47d36e0a" containerName="istio-proxy" Jul 09 15:59:48.102835 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.102789 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce1dba27-b2da-45bc-90db-0e6f47d36e0a" containerName="istio-proxy" Jul 09 15:59:48.102957 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.102933 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce1dba27-b2da-45bc-90db-0e6f47d36e0a" containerName="istio-proxy" Jul 09 15:59:48.105660 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.105641 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.107622 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.107602 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4d56b236-openshift-default-dockercfg-vz2rz\"" Jul 09 15:59:48.125246 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.125224 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2"] Jul 09 15:59:48.216373 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-credential-socket\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216373 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/125edb4f-f47a-4883-b1cf-a143050fe389-istio-podinfo\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-istio-token\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-envoy\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216473 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/125edb4f-f47a-4883-b1cf-a143050fe389-istiod-ca-cert\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-data\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216577 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-socket\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-certs\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.216643 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.216615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr5c6\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-kube-api-access-cr5c6\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317315 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-data\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317510 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-socket\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317510 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-certs\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317646 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr5c6\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-kube-api-access-cr5c6\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317646 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-credential-socket\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-data\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/125edb4f-f47a-4883-b1cf-a143050fe389-istio-podinfo\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317725 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-socket\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317738 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-istio-token\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.317774 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-envoy\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.318025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317809 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/125edb4f-f47a-4883-b1cf-a143050fe389-istiod-ca-cert\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.318025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-certs\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.318025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.317957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-credential-socket\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.318424 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.318402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/125edb4f-f47a-4883-b1cf-a143050fe389-istiod-ca-cert\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.319918 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.319893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-envoy\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.320126 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.320109 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/125edb4f-f47a-4883-b1cf-a143050fe389-istio-podinfo\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.326602 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.326564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-istio-token\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.327025 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.327004 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr5c6\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-kube-api-access-cr5c6\") pod \"e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.415646 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.415615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:48.555843 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.555797 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2"] Jul 09 15:59:48.557181 ip-10-0-133-0 kubenswrapper[2577]: W0709 15:59:48.557147 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125edb4f_f47a_4883_b1cf_a143050fe389.slice/crio-dbe837bb1e35d3306f941066d1a3c724a78c5c4e366a5c4c59b6d8497181c44a WatchSource:0}: Error finding container dbe837bb1e35d3306f941066d1a3c724a78c5c4e366a5c4c59b6d8497181c44a: Status 404 returned error can't find the container with id dbe837bb1e35d3306f941066d1a3c724a78c5c4e366a5c4c59b6d8497181c44a Jul 09 15:59:48.559396 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.559368 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:59:48.559515 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.559442 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:59:48.559515 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:48.559500 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:59:49.297253 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:49.297224 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 15:59:49.430681 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:49.430643 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" event={"ID":"125edb4f-f47a-4883-b1cf-a143050fe389","Type":"ContainerStarted","Data":"be8ef1b86186792bec0de82203b7aeb3c977af5e2928f6b22737d9437432452d"} Jul 09 15:59:49.430860 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:49.430706 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" event={"ID":"125edb4f-f47a-4883-b1cf-a143050fe389","Type":"ContainerStarted","Data":"dbe837bb1e35d3306f941066d1a3c724a78c5c4e366a5c4c59b6d8497181c44a"} Jul 09 15:59:49.455088 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:49.455042 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" podStartSLOduration=1.455028482 podStartE2EDuration="1.455028482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:59:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:59:49.451775579 +0000 UTC m=+1705.989002745" watchObservedRunningTime="2026-07-09 15:59:49.455028482 +0000 UTC m=+1705.992255655" Jul 09 15:59:50.416104 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:50.416075 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:50.420811 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:50.420791 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:50.438128 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:50.438108 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 15:59:50.438930 ip-10-0-133-0 kubenswrapper[2577]: I0709 15:59:50.438913 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:02.397504 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.397457 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:00:02.502813 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.502779 2577 generic.go:356] "Generic (PLEG): container finished" podID="125edb4f-f47a-4883-b1cf-a143050fe389" containerID="be8ef1b86186792bec0de82203b7aeb3c977af5e2928f6b22737d9437432452d" exitCode=0 Jul 09 16:00:02.502968 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.502849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" event={"ID":"125edb4f-f47a-4883-b1cf-a143050fe389","Type":"ContainerDied","Data":"be8ef1b86186792bec0de82203b7aeb3c977af5e2928f6b22737d9437432452d"} Jul 09 16:00:02.503288 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.503273 2577 scope.go:117] "RemoveContainer" containerID="be8ef1b86186792bec0de82203b7aeb3c977af5e2928f6b22737d9437432452d" Jul 09 16:00:02.504272 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.504246 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:02.504340 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.504323 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:02.504385 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:02.504373 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:03.514886 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:03.514842 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" event={"ID":"125edb4f-f47a-4883-b1cf-a143050fe389","Type":"ContainerStarted","Data":"31dddf04081ad51edb28076fecc21e39e2b704a1c2adb2bd541f5b647256da13"} Jul 09 16:00:04.416265 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:04.416232 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:04.417044 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:04.417015 2577 patch_prober.go:28] interesting pod/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 09 16:00:04.417170 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:04.417063 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 09 16:00:05.416141 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:05.416105 2577 patch_prober.go:28] interesting pod/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 09 16:00:05.416641 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:05.416169 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 09 16:00:05.438373 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:05.438291 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:06.416849 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:06.416817 2577 patch_prober.go:28] interesting pod/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 09 16:00:06.417227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:06.416867 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 09 16:00:07.415898 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:07.415864 2577 patch_prober.go:28] interesting pod/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 09 16:00:07.416071 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:07.415913 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 09 16:00:08.420135 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:08.420105 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:08.421075 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:08.421055 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:10.470174 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:10.470137 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2"] Jul 09 16:00:10.470600 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:10.470317 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" containerID="cri-o://31dddf04081ad51edb28076fecc21e39e2b704a1c2adb2bd541f5b647256da13" gracePeriod=30 Jul 09 16:00:13.485235 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:13.485205 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:00:15.587278 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.587251 2577 generic.go:356] "Generic (PLEG): container finished" podID="125edb4f-f47a-4883-b1cf-a143050fe389" containerID="31dddf04081ad51edb28076fecc21e39e2b704a1c2adb2bd541f5b647256da13" exitCode=0 Jul 09 16:00:15.587669 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.587321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" event={"ID":"125edb4f-f47a-4883-b1cf-a143050fe389","Type":"ContainerDied","Data":"31dddf04081ad51edb28076fecc21e39e2b704a1c2adb2bd541f5b647256da13"} Jul 09 16:00:15.587669 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.587380 2577 scope.go:117] "RemoveContainer" containerID="be8ef1b86186792bec0de82203b7aeb3c977af5e2928f6b22737d9437432452d" Jul 09 16:00:15.739060 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.739002 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:15.888520 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888484 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr5c6\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-kube-api-access-cr5c6\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.888707 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888562 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-credential-socket\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.888707 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888585 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-istio-token\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.888707 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888613 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-data\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.888707 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888650 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-socket\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.888707 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888678 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-certs\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.889020 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888730 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/125edb4f-f47a-4883-b1cf-a143050fe389-istiod-ca-cert\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.889020 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888758 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-envoy\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.889020 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888790 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/125edb4f-f47a-4883-b1cf-a143050fe389-istio-podinfo\") pod \"125edb4f-f47a-4883-b1cf-a143050fe389\" (UID: \"125edb4f-f47a-4883-b1cf-a143050fe389\") " Jul 09 16:00:15.889020 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888892 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:00:15.889020 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888936 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:00:15.889020 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.888984 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:00:15.889355 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.889195 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.889355 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.889215 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.889355 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.889229 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.889355 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.889283 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/125edb4f-f47a-4883-b1cf-a143050fe389-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 16:00:15.889700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.889662 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-data" (OuterVolumeSpecName: "istio-data") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:00:15.890850 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.890818 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-kube-api-access-cr5c6" (OuterVolumeSpecName: "kube-api-access-cr5c6") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "kube-api-access-cr5c6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:00:15.890850 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.890835 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-istio-token" (OuterVolumeSpecName: "istio-token") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:00:15.891008 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.890903 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:00:15.891008 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.890937 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/125edb4f-f47a-4883-b1cf-a143050fe389-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "125edb4f-f47a-4883-b1cf-a143050fe389" (UID: "125edb4f-f47a-4883-b1cf-a143050fe389"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 16:00:15.990574 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.990495 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/125edb4f-f47a-4883-b1cf-a143050fe389-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.990574 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.990520 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.990574 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.990529 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/125edb4f-f47a-4883-b1cf-a143050fe389-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.990574 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.990537 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr5c6\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-kube-api-access-cr5c6\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.990574 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.990545 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/125edb4f-f47a-4883-b1cf-a143050fe389-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:15.990574 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:15.990553 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/125edb4f-f47a-4883-b1cf-a143050fe389-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:00:16.594150 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:16.594125 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" Jul 09 16:00:16.594584 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:16.594151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2" event={"ID":"125edb4f-f47a-4883-b1cf-a143050fe389","Type":"ContainerDied","Data":"dbe837bb1e35d3306f941066d1a3c724a78c5c4e366a5c4c59b6d8497181c44a"} Jul 09 16:00:16.594584 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:16.594204 2577 scope.go:117] "RemoveContainer" containerID="31dddf04081ad51edb28076fecc21e39e2b704a1c2adb2bd541f5b647256da13" Jul 09 16:00:16.616764 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:16.616737 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2"] Jul 09 16:00:16.622553 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:16.622533 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4d56b236-openshift-default-d74947c99-4rjc2"] Jul 09 16:00:18.136329 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:18.136297 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" path="/var/lib/kubelet/pods/125edb4f-f47a-4883-b1cf-a143050fe389/volumes" Jul 09 16:00:31.179010 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.178972 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n"] Jul 09 16:00:31.179583 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.179565 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" Jul 09 16:00:31.179583 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.179584 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" Jul 09 16:00:31.179746 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.179695 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" Jul 09 16:00:31.179746 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.179706 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" Jul 09 16:00:31.183081 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.183061 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.184983 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.184962 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-378f7a5c-openshift-default-dockercfg-r7fqd\"" Jul 09 16:00:31.194082 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.194057 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n"] Jul 09 16:00:31.225294 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6de9cf9-6420-4d0c-9436-128cc0731c42-istiod-ca-cert\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225438 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225313 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-credential-socket\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225438 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-socket\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225438 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-data\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225438 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-token\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225629 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-envoy\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225629 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj5wp\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-kube-api-access-dj5wp\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225712 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-podinfo\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.225749 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.225727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-certs\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327123 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6de9cf9-6420-4d0c-9436-128cc0731c42-istiod-ca-cert\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327303 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-credential-socket\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327303 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-socket\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327303 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327169 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-data\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327303 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-token\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327303 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327230 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-envoy\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327303 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj5wp\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-kube-api-access-dj5wp\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327657 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327316 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-podinfo\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327657 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327355 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-certs\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327657 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327594 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-socket\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327657 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-credential-socket\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327859 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-data\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.327859 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.327692 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-certs\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.328036 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.328014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6de9cf9-6420-4d0c-9436-128cc0731c42-istiod-ca-cert\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.329805 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.329779 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-envoy\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.329965 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.329944 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-podinfo\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.334992 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.334967 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-token\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.335072 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.335058 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj5wp\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-kube-api-access-dj5wp\") pod \"e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.495337 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.495245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:31.632668 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.632566 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n"] Jul 09 16:00:31.637314 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:00:31.637289 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6de9cf9_6420_4d0c_9436_128cc0731c42.slice/crio-e80aa2edc21015b5d542289dc07b167c301231c65e343de8ab9c5814689d69f6 WatchSource:0}: Error finding container e80aa2edc21015b5d542289dc07b167c301231c65e343de8ab9c5814689d69f6: Status 404 returned error can't find the container with id e80aa2edc21015b5d542289dc07b167c301231c65e343de8ab9c5814689d69f6 Jul 09 16:00:31.639584 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.639555 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:31.639662 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.639612 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:31.639662 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.639642 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:31.677049 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:31.677024 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" event={"ID":"c6de9cf9-6420-4d0c-9436-128cc0731c42","Type":"ContainerStarted","Data":"e80aa2edc21015b5d542289dc07b167c301231c65e343de8ab9c5814689d69f6"} Jul 09 16:00:32.683504 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:32.683447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" event={"ID":"c6de9cf9-6420-4d0c-9436-128cc0731c42","Type":"ContainerStarted","Data":"90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243"} Jul 09 16:00:32.704407 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:32.704358 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" podStartSLOduration=1.704344949 podStartE2EDuration="1.704344949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:00:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:00:32.701440484 +0000 UTC m=+1749.238667656" watchObservedRunningTime="2026-07-09 16:00:32.704344949 +0000 UTC m=+1749.241572122" Jul 09 16:00:33.495594 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:33.495556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:33.500545 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:33.500523 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:33.688804 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:33.688769 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:33.689680 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:33.689664 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:41.681193 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:41.681157 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:00:49.057122 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.057090 2577 patch_prober.go:28] interesting pod/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": read tcp 10.134.0.2:54908->10.134.0.59:15021: read: connection reset by peer" start-of-body= Jul 09 16:00:49.057528 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.057345 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": read tcp 10.134.0.2:54908->10.134.0.59:15021: read: connection reset by peer" Jul 09 16:00:49.785974 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.785943 2577 generic.go:356] "Generic (PLEG): container finished" podID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerID="90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243" exitCode=0 Jul 09 16:00:49.785974 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.785976 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" event={"ID":"c6de9cf9-6420-4d0c-9436-128cc0731c42","Type":"ContainerDied","Data":"90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243"} Jul 09 16:00:49.786336 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.786320 2577 scope.go:117] "RemoveContainer" containerID="90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243" Jul 09 16:00:49.787225 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.787194 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:49.787342 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.787258 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:49.787342 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:49.787290 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:00:50.793586 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:50.793552 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" event={"ID":"c6de9cf9-6420-4d0c-9436-128cc0731c42","Type":"ContainerStarted","Data":"425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1"} Jul 09 16:00:51.496082 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:51.496049 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:51.496270 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:51.496213 2577 patch_prober.go:28] interesting pod/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 09 16:00:51.496270 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:51.496257 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 09 16:00:52.496581 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:52.496547 2577 patch_prober.go:28] interesting pod/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 09 16:00:52.496961 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:52.496597 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 09 16:00:53.496555 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:53.496520 2577 patch_prober.go:28] interesting pod/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 09 16:00:53.496729 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:53.496585 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 09 16:00:54.499413 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:54.499386 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:54.499876 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:54.499858 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:00:54.500553 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:00:54.500535 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:01:01.678780 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:01:01.678746 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:01:15.287940 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:01:15.287865 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:01:24.161008 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:01:24.160979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 16:01:24.171057 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:01:24.171034 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 16:01:25.990783 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:01:25.990750 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:01:38.695309 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:01:38.695274 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:02:12.899296 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:12.899264 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:02:17.302409 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:17.302376 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n"] Jul 09 16:02:17.302831 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:17.302597 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" containerID="cri-o://425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1" gracePeriod=30 Jul 09 16:02:22.549987 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.549963 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:02:22.611541 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611453 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-data\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611541 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611516 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6de9cf9-6420-4d0c-9436-128cc0731c42-istiod-ca-cert\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611541 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611536 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-credential-socket\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611788 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611571 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj5wp\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-kube-api-access-dj5wp\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611788 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611598 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-token\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611788 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611700 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-envoy\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611788 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611765 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-certs\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611983 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-socket\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.611983 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-podinfo\") pod \"c6de9cf9-6420-4d0c-9436-128cc0731c42\" (UID: \"c6de9cf9-6420-4d0c-9436-128cc0731c42\") " Jul 09 16:02:22.612183 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611853 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:02:22.612183 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.612026 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:02:22.612183 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.611966 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6de9cf9-6420-4d0c-9436-128cc0731c42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 16:02:22.612360 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.612196 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:02:22.612429 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.612403 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:02:22.613846 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.613813 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:02:22.613965 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.613935 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 16:02:22.614062 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.614048 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:02:22.614120 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.614107 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-kube-api-access-dj5wp" (OuterVolumeSpecName: "kube-api-access-dj5wp") pod "c6de9cf9-6420-4d0c-9436-128cc0731c42" (UID: "c6de9cf9-6420-4d0c-9436-128cc0731c42"). InnerVolumeSpecName "kube-api-access-dj5wp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:02:22.712947 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712920 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6de9cf9-6420-4d0c-9436-128cc0731c42-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.712947 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712942 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.712947 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712951 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj5wp\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-kube-api-access-dj5wp\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.713145 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712961 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.713145 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712969 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.713145 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712976 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.713145 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712983 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.713145 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712992 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:22.713145 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:22.712999 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6de9cf9-6420-4d0c-9436-128cc0731c42-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:02:23.303767 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.303738 2577 generic.go:356] "Generic (PLEG): container finished" podID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerID="425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1" exitCode=0 Jul 09 16:02:23.303932 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.303790 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" Jul 09 16:02:23.303932 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.303830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" event={"ID":"c6de9cf9-6420-4d0c-9436-128cc0731c42","Type":"ContainerDied","Data":"425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1"} Jul 09 16:02:23.303932 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.303889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n" event={"ID":"c6de9cf9-6420-4d0c-9436-128cc0731c42","Type":"ContainerDied","Data":"e80aa2edc21015b5d542289dc07b167c301231c65e343de8ab9c5814689d69f6"} Jul 09 16:02:23.303932 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.303918 2577 scope.go:117] "RemoveContainer" containerID="425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1" Jul 09 16:02:23.313454 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.313437 2577 scope.go:117] "RemoveContainer" containerID="90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243" Jul 09 16:02:23.324930 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.324909 2577 scope.go:117] "RemoveContainer" containerID="425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1" Jul 09 16:02:23.325168 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:02:23.325149 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1\": container with ID starting with 425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1 not found: ID does not exist" containerID="425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1" Jul 09 16:02:23.325226 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.325209 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1"} err="failed to get container status \"425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1\": rpc error: code = NotFound desc = could not find container \"425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1\": container with ID starting with 425af00f25aea8ad112a2df00bf59a01322cb654a4105862a72a49753a78a0f1 not found: ID does not exist" Jul 09 16:02:23.325272 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.325229 2577 scope.go:117] "RemoveContainer" containerID="90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243" Jul 09 16:02:23.325524 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:02:23.325507 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243\": container with ID starting with 90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243 not found: ID does not exist" containerID="90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243" Jul 09 16:02:23.325569 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.325529 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243"} err="failed to get container status \"90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243\": rpc error: code = NotFound desc = could not find container \"90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243\": container with ID starting with 90ca05ee468b2b2044e8b54a6d2c7e5501568505350132e3722cf1f499045243 not found: ID does not exist" Jul 09 16:02:23.329160 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.329137 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n"] Jul 09 16:02:23.337600 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:23.337578 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-378f7a5c-openshift-default-bbd8bfdbc-9nm9n"] Jul 09 16:02:24.136821 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:24.136791 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" path="/var/lib/kubelet/pods/c6de9cf9-6420-4d0c-9436-128cc0731c42/volumes" Jul 09 16:02:27.356661 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:27.356626 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:02:36.891651 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:02:36.891618 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:03:14.409604 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.409568 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9"] Jul 09 16:03:14.410109 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.410008 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" Jul 09 16:03:14.410109 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.410025 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="125edb4f-f47a-4883-b1cf-a143050fe389" containerName="istio-proxy" Jul 09 16:03:14.410109 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.410054 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" Jul 09 16:03:14.410109 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.410064 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" Jul 09 16:03:14.410257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.410149 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" Jul 09 16:03:14.413293 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.413278 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.415050 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.415030 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-35b5e6-openshift-default-dockercfg-hlzkm\"" Jul 09 16:03:14.425094 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.425064 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9"] Jul 09 16:03:14.496194 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istiod-ca-cert\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496362 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-data\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496362 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-token\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496362 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496324 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-certs\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496362 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-socket\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496614 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-podinfo\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496684 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496613 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-credential-socket\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496684 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-envoy\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.496794 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.496768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hh2r\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-kube-api-access-7hh2r\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598134 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598107 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-socket\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598302 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-podinfo\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598302 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598183 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-credential-socket\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598302 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-envoy\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598302 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hh2r\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-kube-api-access-7hh2r\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598531 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598380 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istiod-ca-cert\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598531 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-data\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598531 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-token\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598710 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598568 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-socket\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598710 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598585 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-credential-socket\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598710 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-certs\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598861 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-data\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.598931 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.598911 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-certs\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.599086 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.599069 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istiod-ca-cert\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.600456 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.600435 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-envoy\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.600694 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.600676 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-podinfo\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.607655 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.607633 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-token\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.607811 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.607788 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hh2r\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-kube-api-access-7hh2r\") pod \"e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.725891 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.725812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:14.866153 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.866123 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9"] Jul 09 16:03:14.867085 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:03:14.867051 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617c1013_6a63_4a4d_936b_3ccf2f5f15b6.slice/crio-9f6186c704d4e7730d051ead4a321d501ba032624e242124f8d22192bfcad9b3 WatchSource:0}: Error finding container 9f6186c704d4e7730d051ead4a321d501ba032624e242124f8d22192bfcad9b3: Status 404 returned error can't find the container with id 9f6186c704d4e7730d051ead4a321d501ba032624e242124f8d22192bfcad9b3 Jul 09 16:03:14.869157 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.869125 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:03:14.869243 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.869187 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:03:14.869243 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:14.869218 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:03:15.592043 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:15.592010 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" event={"ID":"617c1013-6a63-4a4d-936b-3ccf2f5f15b6","Type":"ContainerStarted","Data":"b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e"} Jul 09 16:03:15.592437 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:15.592071 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" event={"ID":"617c1013-6a63-4a4d-936b-3ccf2f5f15b6","Type":"ContainerStarted","Data":"9f6186c704d4e7730d051ead4a321d501ba032624e242124f8d22192bfcad9b3"} Jul 09 16:03:15.614052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:15.613997 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podStartSLOduration=1.6139783680000002 podStartE2EDuration="1.613978368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:03:15.613410664 +0000 UTC m=+1912.150637836" watchObservedRunningTime="2026-07-09 16:03:15.613978368 +0000 UTC m=+1912.151205540" Jul 09 16:03:15.726080 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:15.726045 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:15.733602 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:15.733575 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:16.598634 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:16.598602 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:16.599559 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:16.599539 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:25.804485 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:25.804424 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:03:32.639691 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:32.639651 2577 patch_prober.go:28] interesting pod/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 16:03:32.640133 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:32.639723 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 16:03:33.696378 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:33.696346 2577 generic.go:356] "Generic (PLEG): container finished" podID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerID="b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e" exitCode=0 Jul 09 16:03:33.696787 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:33.696419 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" event={"ID":"617c1013-6a63-4a4d-936b-3ccf2f5f15b6","Type":"ContainerDied","Data":"b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e"} Jul 09 16:03:33.696841 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:33.696826 2577 scope.go:117] "RemoveContainer" containerID="b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e" Jul 09 16:03:33.697842 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:33.697816 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:03:33.697910 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:33.697897 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:03:33.697951 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:33.697939 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:03:34.705035 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:34.705001 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" event={"ID":"617c1013-6a63-4a4d-936b-3ccf2f5f15b6","Type":"ContainerStarted","Data":"2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05"} Jul 09 16:03:34.729377 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:34.729344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:34.729942 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:34.729576 2577 patch_prober.go:28] interesting pod/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 09 16:03:34.729942 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:34.729724 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 09 16:03:35.726741 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:35.726704 2577 patch_prober.go:28] interesting pod/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 09 16:03:35.727321 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:35.727291 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 09 16:03:36.625294 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.625243 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp"] Jul 09 16:03:36.625911 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.625882 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" Jul 09 16:03:36.625911 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.625910 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" Jul 09 16:03:36.626098 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.626037 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6de9cf9-6420-4d0c-9436-128cc0731c42" containerName="istio-proxy" Jul 09 16:03:36.629173 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.629151 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.632221 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.632200 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-35b5e6\"/\"default-dockercfg-f6lss\"" Jul 09 16:03:36.636322 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.635514 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-35b5e6\"/\"test-model-35b5e6-kserve-self-signed-certs\"" Jul 09 16:03:36.636322 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.635759 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-35b5e6\"/\"kube-root-ca.crt\"" Jul 09 16:03:36.636322 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.636138 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-35b5e6\"/\"openshift-service-ca.crt\"" Jul 09 16:03:36.650057 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.650021 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp"] Jul 09 16:03:36.721456 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.721418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tmp-dir\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.721655 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.721492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-model-cache\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.721655 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.721525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tls-certs\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.721655 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.721563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-home\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.721800 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.721671 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlwnf\" (UniqueName: \"kubernetes.io/projected/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-kube-api-access-dlwnf\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.721800 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.721725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-dshm\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.728197 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.727797 2577 patch_prober.go:28] interesting pod/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 09 16:03:36.728197 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.727851 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 09 16:03:36.823234 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.823193 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlwnf\" (UniqueName: \"kubernetes.io/projected/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-kube-api-access-dlwnf\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.823423 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.823250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-dshm\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.823423 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.823291 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tmp-dir\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.823423 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.823327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-model-cache\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.823632 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.823441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tls-certs\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.823759 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.823737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-model-cache\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.824226 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.824206 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-home\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.824426 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.824409 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tmp-dir\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.835513 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.835488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-home\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.845755 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.845725 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-dshm\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.847117 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.847083 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlwnf\" (UniqueName: \"kubernetes.io/projected/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-kube-api-access-dlwnf\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.854481 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.854038 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tls-certs\") pod \"test-model-35b5e6-kserve-ff76d5789-znrdp\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:36.945747 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:36.945645 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:37.112079 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:37.112056 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp"] Jul 09 16:03:37.113794 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:03:37.113772 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4d98bf4_4bf4_4a5f_96b6_7ff3c4f2554d.slice/crio-65718e0ee921d61762abdeae502b409c44df8b1471bcddcdf5ce69a4c51419b5 WatchSource:0}: Error finding container 65718e0ee921d61762abdeae502b409c44df8b1471bcddcdf5ce69a4c51419b5: Status 404 returned error can't find the container with id 65718e0ee921d61762abdeae502b409c44df8b1471bcddcdf5ce69a4c51419b5 Jul 09 16:03:37.723047 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:37.723015 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" event={"ID":"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d","Type":"ContainerStarted","Data":"9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904"} Jul 09 16:03:37.723228 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:37.723060 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" event={"ID":"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d","Type":"ContainerStarted","Data":"65718e0ee921d61762abdeae502b409c44df8b1471bcddcdf5ce69a4c51419b5"} Jul 09 16:03:37.726678 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:37.726643 2577 patch_prober.go:28] interesting pod/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 09 16:03:37.726834 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:37.726706 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 09 16:03:37.752581 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:37.752524 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" podStartSLOduration=1.7525071620000001 podStartE2EDuration="1.752507162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:03:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:03:37.751188893 +0000 UTC m=+1934.288416067" watchObservedRunningTime="2026-07-09 16:03:37.752507162 +0000 UTC m=+1934.289734336" Jul 09 16:03:38.730383 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:38.730355 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:38.730857 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:38.730840 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:38.731550 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:38.731534 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:03:41.305914 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:41.305877 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:03:46.946037 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:46.946003 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:46.946037 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:46.946046 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:46.958794 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:46.958771 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:47.788087 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:47.788059 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:03:55.127695 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:03:55.127658 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:04:11.205161 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:11.205126 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:04:44.431396 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:44.431350 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp"] Jul 09 16:04:49.489787 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:49.489754 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:04:59.451865 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.451807 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" podUID="f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" containerName="main" containerID="cri-o://9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904" gracePeriod=45 Jul 09 16:04:59.698739 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.698709 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:04:59.824887 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.824782 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tmp-dir\") pod \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " Jul 09 16:04:59.824887 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.824845 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlwnf\" (UniqueName: \"kubernetes.io/projected/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-kube-api-access-dlwnf\") pod \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " Jul 09 16:04:59.824887 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.824884 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-model-cache\") pod \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " Jul 09 16:04:59.825193 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.824909 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-dshm\") pod \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " Jul 09 16:04:59.825193 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.824956 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-home\") pod \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " Jul 09 16:04:59.825193 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825076 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tls-certs\") pod \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\" (UID: \"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d\") " Jul 09 16:04:59.825193 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825112 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" (UID: "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:04:59.825193 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825164 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-model-cache" (OuterVolumeSpecName: "model-cache") pod "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" (UID: "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:04:59.825406 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825268 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-home" (OuterVolumeSpecName: "home") pod "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" (UID: "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:04:59.825449 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825427 2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tmp-dir\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:04:59.825449 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825444 2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-model-cache\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:04:59.825617 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.825457 2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-home\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:04:59.827123 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.827093 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" (UID: "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 16:04:59.827123 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.827106 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-kube-api-access-dlwnf" (OuterVolumeSpecName: "kube-api-access-dlwnf") pod "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" (UID: "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d"). InnerVolumeSpecName "kube-api-access-dlwnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:04:59.827314 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.827121 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-dshm" (OuterVolumeSpecName: "dshm") pod "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" (UID: "f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:04:59.926288 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.926258 2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-tls-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:04:59.926288 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.926284 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlwnf\" (UniqueName: \"kubernetes.io/projected/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-kube-api-access-dlwnf\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:04:59.926288 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:04:59.926294 2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d-dshm\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:00.190959 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.190931 2577 generic.go:356] "Generic (PLEG): container finished" podID="f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" containerID="9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904" exitCode=0 Jul 09 16:05:00.191119 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.190973 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" Jul 09 16:05:00.191119 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.191020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" event={"ID":"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d","Type":"ContainerDied","Data":"9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904"} Jul 09 16:05:00.191119 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.191056 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp" event={"ID":"f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d","Type":"ContainerDied","Data":"65718e0ee921d61762abdeae502b409c44df8b1471bcddcdf5ce69a4c51419b5"} Jul 09 16:05:00.191119 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.191078 2577 scope.go:117] "RemoveContainer" containerID="9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904" Jul 09 16:05:00.200237 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.200221 2577 scope.go:117] "RemoveContainer" containerID="9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904" Jul 09 16:05:00.200514 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:05:00.200496 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904\": container with ID starting with 9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904 not found: ID does not exist" containerID="9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904" Jul 09 16:05:00.200595 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.200520 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904"} err="failed to get container status \"9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904\": rpc error: code = NotFound desc = could not find container \"9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904\": container with ID starting with 9b28f95f5f7c4fa6a5f2dcad259a6f035050ab8645e8ae009e8d9b53486cc904 not found: ID does not exist" Jul 09 16:05:00.214513 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.214490 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp"] Jul 09 16:05:00.217571 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:00.217551 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-35b5e6/test-model-35b5e6-kserve-ff76d5789-znrdp"] Jul 09 16:05:02.137758 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:02.137723 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" path="/var/lib/kubelet/pods/f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d/volumes" Jul 09 16:05:29.309134 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:29.309100 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9"] Jul 09 16:05:29.309960 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:29.309324 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" containerID="cri-o://2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05" gracePeriod=30 Jul 09 16:05:33.481047 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:33.481011 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:05:34.568622 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.568599 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:05:34.646249 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646220 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-token\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646430 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646279 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hh2r\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-kube-api-access-7hh2r\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646430 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646329 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-credential-socket\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646430 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-podinfo\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646430 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646392 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istiod-ca-cert\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646682 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646440 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-certs\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646682 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646491 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-envoy\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646682 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646538 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-socket\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646682 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646564 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-data\") pod \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\" (UID: \"617c1013-6a63-4a4d-936b-3ccf2f5f15b6\") " Jul 09 16:05:34.646889 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646782 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:05:34.646972 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.646948 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:05:34.647073 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.647018 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 16:05:34.647073 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.647063 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.647310 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.647083 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.647310 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.647226 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:05:34.647923 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.647899 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:05:34.648867 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.648846 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:05:34.648956 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.648939 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:05:34.649097 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.649072 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 16:05:34.649231 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.649094 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-kube-api-access-7hh2r" (OuterVolumeSpecName: "kube-api-access-7hh2r") pod "617c1013-6a63-4a4d-936b-3ccf2f5f15b6" (UID: "617c1013-6a63-4a4d-936b-3ccf2f5f15b6"). InnerVolumeSpecName "kube-api-access-7hh2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:05:34.747829 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747792 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hh2r\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-kube-api-access-7hh2r\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.747829 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747823 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.747829 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747832 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.748069 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747843 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.748069 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747851 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.748069 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747858 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:34.748069 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:34.747865 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617c1013-6a63-4a4d-936b-3ccf2f5f15b6-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:05:35.388074 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.388045 2577 generic.go:356] "Generic (PLEG): container finished" podID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerID="2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05" exitCode=0 Jul 09 16:05:35.388257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.388092 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" Jul 09 16:05:35.388257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.388116 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" event={"ID":"617c1013-6a63-4a4d-936b-3ccf2f5f15b6","Type":"ContainerDied","Data":"2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05"} Jul 09 16:05:35.388257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.388162 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9" event={"ID":"617c1013-6a63-4a4d-936b-3ccf2f5f15b6","Type":"ContainerDied","Data":"9f6186c704d4e7730d051ead4a321d501ba032624e242124f8d22192bfcad9b3"} Jul 09 16:05:35.388257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.388184 2577 scope.go:117] "RemoveContainer" containerID="2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05" Jul 09 16:05:35.397657 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.397639 2577 scope.go:117] "RemoveContainer" containerID="b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e" Jul 09 16:05:35.405428 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.405408 2577 scope.go:117] "RemoveContainer" containerID="2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05" Jul 09 16:05:35.405687 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:05:35.405667 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05\": container with ID starting with 2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05 not found: ID does not exist" containerID="2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05" Jul 09 16:05:35.405763 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.405702 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05"} err="failed to get container status \"2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05\": rpc error: code = NotFound desc = could not find container \"2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05\": container with ID starting with 2fe1b593bfd3d4bde7f9b290efcade89487a7fd67acd572da765321a56568c05 not found: ID does not exist" Jul 09 16:05:35.405763 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.405719 2577 scope.go:117] "RemoveContainer" containerID="b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e" Jul 09 16:05:35.405957 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:05:35.405940 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e\": container with ID starting with b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e not found: ID does not exist" containerID="b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e" Jul 09 16:05:35.405997 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.405964 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e"} err="failed to get container status \"b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e\": rpc error: code = NotFound desc = could not find container \"b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e\": container with ID starting with b6456af258c9c0305f42db3404bc8d67b97d389e6fea0e650e9e9e31e59d509e not found: ID does not exist" Jul 09 16:05:35.410602 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.410580 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9"] Jul 09 16:05:35.415386 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:35.415361 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-35b5e6-openshift-default-579f5dcc69-h7pl9"] Jul 09 16:05:36.136177 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:36.136146 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" path="/var/lib/kubelet/pods/617c1013-6a63-4a4d-936b-3ccf2f5f15b6/volumes" Jul 09 16:05:37.913260 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913229 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg"] Jul 09 16:05:37.913685 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913671 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" Jul 09 16:05:37.913750 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913687 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" Jul 09 16:05:37.913750 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913709 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" containerName="main" Jul 09 16:05:37.913750 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913715 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" containerName="main" Jul 09 16:05:37.913880 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913796 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" Jul 09 16:05:37.913880 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913807 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" Jul 09 16:05:37.913880 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.913815 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d98bf4-4bf4-4a5f-96b6-7ff3c4f2554d" containerName="main" Jul 09 16:05:37.917262 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.917239 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:37.919148 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.919129 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5363ce-openshift-default-dockercfg-svtl8\"" Jul 09 16:05:37.929953 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:37.929923 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg"] Jul 09 16:05:38.083999 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.083961 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-credential-socket\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.083999 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-socket\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084039 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-certs\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084070 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-token\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4bvf\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-kube-api-access-f4bvf\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-podinfo\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-data\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39348dbc-ff20-4f96-8c81-9facb77cc765-istiod-ca-cert\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.084236 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.084201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-envoy\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185301 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-token\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185301 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4bvf\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-kube-api-access-f4bvf\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185301 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-podinfo\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185301 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-data\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185668 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39348dbc-ff20-4f96-8c81-9facb77cc765-istiod-ca-cert\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185668 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185355 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-envoy\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185668 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-credential-socket\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185668 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185412 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-socket\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185668 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-certs\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185939 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-credential-socket\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185997 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185938 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-data\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.185997 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.185964 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-certs\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.186120 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.186074 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-socket\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.186224 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.186202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39348dbc-ff20-4f96-8c81-9facb77cc765-istiod-ca-cert\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.187783 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.187762 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-podinfo\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.187874 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.187835 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-envoy\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.193743 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.193720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-token\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.193841 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.193790 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4bvf\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-kube-api-access-f4bvf\") pod \"e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.228871 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.228846 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:38.370861 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.370828 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg"] Jul 09 16:05:38.371985 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:05:38.371961 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39348dbc_ff20_4f96_8c81_9facb77cc765.slice/crio-c8fa485131fbad61fdd1d95e19367069a08f5f831f493bb1b04d877d763afbd1 WatchSource:0}: Error finding container c8fa485131fbad61fdd1d95e19367069a08f5f831f493bb1b04d877d763afbd1: Status 404 returned error can't find the container with id c8fa485131fbad61fdd1d95e19367069a08f5f831f493bb1b04d877d763afbd1 Jul 09 16:05:38.374034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.374002 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:05:38.374122 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.374066 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:05:38.374122 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.374093 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:05:38.407977 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:38.407941 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" event={"ID":"39348dbc-ff20-4f96-8c81-9facb77cc765","Type":"ContainerStarted","Data":"c8fa485131fbad61fdd1d95e19367069a08f5f831f493bb1b04d877d763afbd1"} Jul 09 16:05:39.415556 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:39.415515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" event={"ID":"39348dbc-ff20-4f96-8c81-9facb77cc765","Type":"ContainerStarted","Data":"f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030"} Jul 09 16:05:39.437711 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:39.437652 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" podStartSLOduration=2.437636455 podStartE2EDuration="2.437636455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:05:39.435742126 +0000 UTC m=+2055.972969299" watchObservedRunningTime="2026-07-09 16:05:39.437636455 +0000 UTC m=+2055.974863628" Jul 09 16:05:40.229771 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:40.229735 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:40.234878 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:40.234849 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:40.421164 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:40.421136 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:40.422101 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:40.422082 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:46.612684 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:46.612610 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:05:54.503077 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:54.502996 2577 generic.go:356] "Generic (PLEG): container finished" podID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerID="f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030" exitCode=0 Jul 09 16:05:54.503539 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:54.503070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" event={"ID":"39348dbc-ff20-4f96-8c81-9facb77cc765","Type":"ContainerDied","Data":"f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030"} Jul 09 16:05:54.503539 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:54.503454 2577 scope.go:117] "RemoveContainer" containerID="f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030" Jul 09 16:05:54.504449 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:54.504424 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:05:54.504532 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:54.504502 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:05:54.504571 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:54.504532 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:05:55.421246 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:55.421212 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:55.510802 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:55.510756 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" event={"ID":"39348dbc-ff20-4f96-8c81-9facb77cc765","Type":"ContainerStarted","Data":"2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e"} Jul 09 16:05:56.228988 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:56.228948 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:05:56.230520 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:56.230489 2577 patch_prober.go:28] interesting pod/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 09 16:05:56.230659 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:56.230562 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 09 16:05:57.229406 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:57.229364 2577 patch_prober.go:28] interesting pod/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 09 16:05:57.229929 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:57.229432 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 09 16:05:58.229878 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:58.229838 2577 patch_prober.go:28] interesting pod/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 09 16:05:58.230358 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:58.229915 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 09 16:05:59.229706 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:59.229660 2577 patch_prober.go:28] interesting pod/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 09 16:05:59.229919 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:05:59.229741 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 09 16:06:00.233513 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:00.233485 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:06:00.233895 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:00.233824 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:06:00.234490 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:00.234452 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:06:02.597841 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:02.597809 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:06:09.215700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:09.215665 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg"] Jul 09 16:06:09.216114 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:09.215891 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" containerID="cri-o://2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e" gracePeriod=30 Jul 09 16:06:12.783037 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:12.783001 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:06:14.470706 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.470683 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:06:14.523412 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523336 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-data\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523412 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523388 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-certs\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523415 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-podinfo\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523440 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-socket\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523478 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4bvf\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-kube-api-access-f4bvf\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523537 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39348dbc-ff20-4f96-8c81-9facb77cc765-istiod-ca-cert\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523575 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-envoy\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523609 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-credential-socket\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523941 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523676 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-token\") pod \"39348dbc-ff20-4f96-8c81-9facb77cc765\" (UID: \"39348dbc-ff20-4f96-8c81-9facb77cc765\") " Jul 09 16:06:14.523941 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.523758 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:06:14.524062 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.524012 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.524257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.524219 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:06:14.524257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.524244 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-data" (OuterVolumeSpecName: "istio-data") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:06:14.524706 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.524646 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39348dbc-ff20-4f96-8c81-9facb77cc765-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 16:06:14.524895 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.524856 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:06:14.526139 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.526079 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:06:14.526556 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.526532 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 16:06:14.526556 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.526547 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-token" (OuterVolumeSpecName: "istio-token") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:06:14.527044 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.527020 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-kube-api-access-f4bvf" (OuterVolumeSpecName: "kube-api-access-f4bvf") pod "39348dbc-ff20-4f96-8c81-9facb77cc765" (UID: "39348dbc-ff20-4f96-8c81-9facb77cc765"). InnerVolumeSpecName "kube-api-access-f4bvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:06:14.619211 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.619180 2577 generic.go:356] "Generic (PLEG): container finished" podID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerID="2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e" exitCode=0 Jul 09 16:06:14.619375 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.619222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" event={"ID":"39348dbc-ff20-4f96-8c81-9facb77cc765","Type":"ContainerDied","Data":"2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e"} Jul 09 16:06:14.619375 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.619234 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" Jul 09 16:06:14.619375 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.619273 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg" event={"ID":"39348dbc-ff20-4f96-8c81-9facb77cc765","Type":"ContainerDied","Data":"c8fa485131fbad61fdd1d95e19367069a08f5f831f493bb1b04d877d763afbd1"} Jul 09 16:06:14.619375 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.619300 2577 scope.go:117] "RemoveContainer" containerID="2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e" Jul 09 16:06:14.624943 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.624920 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39348dbc-ff20-4f96-8c81-9facb77cc765-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.624947 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.624961 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.624975 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.624988 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.624999 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39348dbc-ff20-4f96-8c81-9facb77cc765-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.625012 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39348dbc-ff20-4f96-8c81-9facb77cc765-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.625034 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.625027 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4bvf\" (UniqueName: \"kubernetes.io/projected/39348dbc-ff20-4f96-8c81-9facb77cc765-kube-api-access-f4bvf\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:06:14.629716 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.629605 2577 scope.go:117] "RemoveContainer" containerID="f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030" Jul 09 16:06:14.639201 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.639187 2577 scope.go:117] "RemoveContainer" containerID="2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e" Jul 09 16:06:14.639423 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:06:14.639405 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e\": container with ID starting with 2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e not found: ID does not exist" containerID="2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e" Jul 09 16:06:14.639617 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.639434 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e"} err="failed to get container status \"2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e\": rpc error: code = NotFound desc = could not find container \"2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e\": container with ID starting with 2a25e2f124f2b9a739167569383a27055cb2b7d7f32514f19066cbd4f235478e not found: ID does not exist" Jul 09 16:06:14.639617 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.639472 2577 scope.go:117] "RemoveContainer" containerID="f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030" Jul 09 16:06:14.639997 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:06:14.639933 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030\": container with ID starting with f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030 not found: ID does not exist" containerID="f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030" Jul 09 16:06:14.640094 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.640007 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030"} err="failed to get container status \"f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030\": rpc error: code = NotFound desc = could not find container \"f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030\": container with ID starting with f7d8bae628a2d379f8f02cd7bfdeeea55780ed0c6c8de2271ba8728a30a64030 not found: ID does not exist" Jul 09 16:06:14.643627 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.643605 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg"] Jul 09 16:06:14.646233 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:14.646214 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5363ce-openshift-default-54bfb65769-5hzwg"] Jul 09 16:06:16.137604 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:16.137577 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" path="/var/lib/kubelet/pods/39348dbc-ff20-4f96-8c81-9facb77cc765/volumes" Jul 09 16:06:17.764013 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.763984 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq"] Jul 09 16:06:17.764396 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764383 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" Jul 09 16:06:17.764396 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764397 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="617c1013-6a63-4a4d-936b-3ccf2f5f15b6" containerName="istio-proxy" Jul 09 16:06:17.764487 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764414 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" Jul 09 16:06:17.764487 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764419 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" Jul 09 16:06:17.764487 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764437 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" Jul 09 16:06:17.764487 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764443 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" Jul 09 16:06:17.764616 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764533 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" Jul 09 16:06:17.764616 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.764541 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39348dbc-ff20-4f96-8c81-9facb77cc765" containerName="istio-proxy" Jul 09 16:06:17.768154 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.768136 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.770936 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.770915 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c8e6dd-openshift-default-dockercfg-8srfs\"" Jul 09 16:06:17.783956 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.783935 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq"] Jul 09 16:06:17.851689 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851664 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-istio-token\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.851844 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-data\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.851844 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851756 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-certs\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.851844 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7167618b-bd7b-488d-b795-1306579a2420-istiod-ca-cert\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.851844 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpgv\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-kube-api-access-7kpgv\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.852052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851925 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-socket\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.852052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.851963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-envoy\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.852052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.852028 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7167618b-bd7b-488d-b795-1306579a2420-istio-podinfo\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.852162 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.852070 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-credential-socket\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.953524 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.953492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-socket\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.953969 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.953939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-envoy\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954085 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7167618b-bd7b-488d-b795-1306579a2420-istio-podinfo\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954085 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954054 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-credential-socket\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954192 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954130 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-istio-token\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954192 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954169 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-data\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954299 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-certs\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954299 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7167618b-bd7b-488d-b795-1306579a2420-istiod-ca-cert\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954299 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpgv\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-kube-api-access-7kpgv\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.954888 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954623 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-credential-socket\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.955012 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.954948 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-data\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.955085 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.955009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-certs\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.955270 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.955238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7167618b-bd7b-488d-b795-1306579a2420-istiod-ca-cert\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.955358 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.955238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-socket\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.957237 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.957210 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-envoy\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.957506 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.957488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7167618b-bd7b-488d-b795-1306579a2420-istio-podinfo\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.962453 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.962425 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-istio-token\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:17.962700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:17.962684 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpgv\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-kube-api-access-7kpgv\") pod \"e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:18.078967 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.078901 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:18.224312 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.224286 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq"] Jul 09 16:06:18.225645 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:06:18.225616 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7167618b_bd7b_488d_b795_1306579a2420.slice/crio-0957af15b02eba98c2d5ab2b7032415cba35a229f55a857c233525440b30c5ca WatchSource:0}: Error finding container 0957af15b02eba98c2d5ab2b7032415cba35a229f55a857c233525440b30c5ca: Status 404 returned error can't find the container with id 0957af15b02eba98c2d5ab2b7032415cba35a229f55a857c233525440b30c5ca Jul 09 16:06:18.227831 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.227802 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:18.227927 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.227877 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:18.227989 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.227923 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:18.645069 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.645033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" event={"ID":"7167618b-bd7b-488d-b795-1306579a2420","Type":"ContainerStarted","Data":"5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e"} Jul 09 16:06:18.645250 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.645092 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" event={"ID":"7167618b-bd7b-488d-b795-1306579a2420","Type":"ContainerStarted","Data":"0957af15b02eba98c2d5ab2b7032415cba35a229f55a857c233525440b30c5ca"} Jul 09 16:06:18.664831 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:18.664766 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" podStartSLOduration=1.664751181 podStartE2EDuration="1.664751181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:06:18.664039151 +0000 UTC m=+2095.201266323" watchObservedRunningTime="2026-07-09 16:06:18.664751181 +0000 UTC m=+2095.201978385" Jul 09 16:06:19.079079 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:19.078997 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:19.084021 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:19.083998 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:19.650839 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:19.650807 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:19.651793 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:19.651775 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:24.211397 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:24.211367 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 16:06:24.215946 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:24.215927 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pj2r4_547d60d5-feb0-45c1-af93-9eb6d5b593c4/ovn-acl-logging/0.log" Jul 09 16:06:25.495230 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.495192 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:06:25.791960 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.791880 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb"] Jul 09 16:06:25.795751 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.795726 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.797621 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.797600 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1a4b78-openshift-default-dockercfg-m82lm\"" Jul 09 16:06:25.812440 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.812418 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb"] Jul 09 16:06:25.930108 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-certs\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930297 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-token\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930297 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-socket\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930297 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-credential-socket\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930424 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-podinfo\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930424 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-envoy\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930424 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istiod-ca-cert\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930552 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930440 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp92s\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-kube-api-access-cp92s\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:25.930552 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:25.930506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-data\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.031870 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.031833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istiod-ca-cert\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.031892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp92s\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-kube-api-access-cp92s\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.031947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-data\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.031987 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-certs\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032056 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-token\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-socket\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032122 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-credential-socket\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-podinfo\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-envoy\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032962 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032650 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istiod-ca-cert\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032962 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032798 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-certs\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.032962 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.032875 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-socket\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.033202 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.033159 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-data\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.033521 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.033502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-credential-socket\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.034782 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.034756 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-envoy\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.035392 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.035369 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-podinfo\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.041250 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.041228 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-token\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.041524 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.041506 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp92s\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-kube-api-access-cp92s\") pod \"e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.108642 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.108571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:26.263323 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.263295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb"] Jul 09 16:06:26.265369 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:06:26.265342 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20680a9_6af1_401a_83f4_b8bfb7db2b2a.slice/crio-796f25c401945c6a4d2502bcdc617e350f74831b45171aed05df1f71053c4753 WatchSource:0}: Error finding container 796f25c401945c6a4d2502bcdc617e350f74831b45171aed05df1f71053c4753: Status 404 returned error can't find the container with id 796f25c401945c6a4d2502bcdc617e350f74831b45171aed05df1f71053c4753 Jul 09 16:06:26.268026 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.267994 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:26.268104 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.268066 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:26.268142 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.268109 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:26.692310 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.692275 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" event={"ID":"b20680a9-6af1-401a-83f4-b8bfb7db2b2a","Type":"ContainerStarted","Data":"3bbf6d2c5915ea4909d5bfe00b931748e4f7f67d6d143e8f099147344cf9206a"} Jul 09 16:06:26.692716 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.692326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" event={"ID":"b20680a9-6af1-401a-83f4-b8bfb7db2b2a","Type":"ContainerStarted","Data":"796f25c401945c6a4d2502bcdc617e350f74831b45171aed05df1f71053c4753"} Jul 09 16:06:26.715733 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:26.715688 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" podStartSLOduration=1.7156748419999999 podStartE2EDuration="1.715674842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:06:26.713642741 +0000 UTC m=+2103.250869914" watchObservedRunningTime="2026-07-09 16:06:26.715674842 +0000 UTC m=+2103.252902015" Jul 09 16:06:27.108964 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:27.108881 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:27.113740 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:27.113718 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:27.699429 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:27.699398 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:27.700556 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:27.700536 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:33.737042 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:33.737002 2577 generic.go:356] "Generic (PLEG): container finished" podID="7167618b-bd7b-488d-b795-1306579a2420" containerID="5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e" exitCode=0 Jul 09 16:06:33.737602 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:33.737151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" event={"ID":"7167618b-bd7b-488d-b795-1306579a2420","Type":"ContainerDied","Data":"5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e"} Jul 09 16:06:33.737720 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:33.737690 2577 scope.go:117] "RemoveContainer" containerID="5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e" Jul 09 16:06:33.740000 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:33.739653 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:33.740000 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:33.739747 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:33.740000 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:33.739789 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:34.650747 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:34.650719 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:34.750879 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:34.750830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" event={"ID":"7167618b-bd7b-488d-b795-1306579a2420","Type":"ContainerStarted","Data":"2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d"} Jul 09 16:06:35.080161 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:35.080071 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:35.081062 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:35.081033 2577 patch_prober.go:28] interesting pod/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 09 16:06:35.081184 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:35.081097 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 09 16:06:36.079847 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:36.079813 2577 patch_prober.go:28] interesting pod/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 09 16:06:36.080246 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:36.079868 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 09 16:06:37.079952 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:37.079913 2577 patch_prober.go:28] interesting pod/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 09 16:06:37.080335 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:37.079985 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 09 16:06:38.079997 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:38.079958 2577 patch_prober.go:28] interesting pod/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 09 16:06:38.080487 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:38.080015 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 09 16:06:39.083620 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:39.083589 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:39.084052 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:39.083847 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:39.084570 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:39.084554 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:06:40.794122 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:40.794089 2577 generic.go:356] "Generic (PLEG): container finished" podID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerID="3bbf6d2c5915ea4909d5bfe00b931748e4f7f67d6d143e8f099147344cf9206a" exitCode=0 Jul 09 16:06:40.794547 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:40.794163 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" event={"ID":"b20680a9-6af1-401a-83f4-b8bfb7db2b2a","Type":"ContainerDied","Data":"3bbf6d2c5915ea4909d5bfe00b931748e4f7f67d6d143e8f099147344cf9206a"} Jul 09 16:06:40.794693 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:40.794678 2577 scope.go:117] "RemoveContainer" containerID="3bbf6d2c5915ea4909d5bfe00b931748e4f7f67d6d143e8f099147344cf9206a" Jul 09 16:06:40.795552 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:40.795530 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:40.795613 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:40.795589 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:40.795650 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:40.795624 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 16:06:41.392448 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:41.392414 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:06:41.802285 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:41.802204 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" event={"ID":"b20680a9-6af1-401a-83f4-b8bfb7db2b2a","Type":"ContainerStarted","Data":"eab809b5eb4721220946658f8e37137bc712c8ad1e34aaac67fd087e1adb2e5a"} Jul 09 16:06:42.109827 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:42.109738 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:42.110933 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:42.110907 2577 patch_prober.go:28] interesting pod/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 09 16:06:42.111067 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:42.110961 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 09 16:06:42.699446 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:42.699410 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:43.108968 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:43.108887 2577 patch_prober.go:28] interesting pod/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 09 16:06:43.108968 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:43.108945 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 09 16:06:44.109920 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:44.109887 2577 patch_prober.go:28] interesting pod/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 09 16:06:44.110311 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:44.109953 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 09 16:06:45.110062 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:45.110026 2577 patch_prober.go:28] interesting pod/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 09 16:06:45.110505 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:45.110077 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 09 16:06:46.113735 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:46.113690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:06:46.114669 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:06:46.114650 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:09:42.484271 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:42.484229 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:09:45.360578 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:45.360545 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq"] Jul 09 16:09:45.361008 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:45.360751 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" containerID="cri-o://2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d" gracePeriod=30 Jul 09 16:09:50.616776 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.616751 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:09:50.713329 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713296 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-certs\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713329 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713331 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-envoy\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713619 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713370 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-socket\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713619 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713388 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-istio-token\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713619 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713407 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7167618b-bd7b-488d-b795-1306579a2420-istio-podinfo\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713619 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713449 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-data\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713619 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713579 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kpgv\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-kube-api-access-7kpgv\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713619 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713613 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7167618b-bd7b-488d-b795-1306579a2420-istiod-ca-cert\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713935 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713634 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-credential-socket\") pod \"7167618b-bd7b-488d-b795-1306579a2420\" (UID: \"7167618b-bd7b-488d-b795-1306579a2420\") " Jul 09 16:09:50.713935 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713661 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:50.713935 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713753 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:50.714103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.713969 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:50.714103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.714014 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.714103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.714034 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.714103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.714071 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7167618b-bd7b-488d-b795-1306579a2420-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 16:09:50.714562 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.714539 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-data" (OuterVolumeSpecName: "istio-data") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:50.715568 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.715537 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:50.715682 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.715666 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-kube-api-access-7kpgv" (OuterVolumeSpecName: "kube-api-access-7kpgv") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "kube-api-access-7kpgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:09:50.715917 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.715896 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-istio-token" (OuterVolumeSpecName: "istio-token") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:09:50.715917 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.715902 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7167618b-bd7b-488d-b795-1306579a2420-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7167618b-bd7b-488d-b795-1306579a2420" (UID: "7167618b-bd7b-488d-b795-1306579a2420"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 16:09:50.814732 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814698 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.814732 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814725 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kpgv\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-kube-api-access-7kpgv\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.814732 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814736 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7167618b-bd7b-488d-b795-1306579a2420-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.814939 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814744 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.814939 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814753 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7167618b-bd7b-488d-b795-1306579a2420-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.814939 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814762 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7167618b-bd7b-488d-b795-1306579a2420-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.814939 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.814768 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7167618b-bd7b-488d-b795-1306579a2420-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:50.880189 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.880158 2577 generic.go:356] "Generic (PLEG): container finished" podID="7167618b-bd7b-488d-b795-1306579a2420" containerID="2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d" exitCode=0 Jul 09 16:09:50.880328 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.880211 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" Jul 09 16:09:50.880328 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.880242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" event={"ID":"7167618b-bd7b-488d-b795-1306579a2420","Type":"ContainerDied","Data":"2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d"} Jul 09 16:09:50.880328 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.880304 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq" event={"ID":"7167618b-bd7b-488d-b795-1306579a2420","Type":"ContainerDied","Data":"0957af15b02eba98c2d5ab2b7032415cba35a229f55a857c233525440b30c5ca"} Jul 09 16:09:50.880453 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.880329 2577 scope.go:117] "RemoveContainer" containerID="2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d" Jul 09 16:09:50.889988 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.889973 2577 scope.go:117] "RemoveContainer" containerID="5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e" Jul 09 16:09:50.897596 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.897581 2577 scope.go:117] "RemoveContainer" containerID="2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d" Jul 09 16:09:50.897835 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:09:50.897818 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d\": container with ID starting with 2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d not found: ID does not exist" containerID="2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d" Jul 09 16:09:50.897896 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.897841 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d"} err="failed to get container status \"2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d\": rpc error: code = NotFound desc = could not find container \"2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d\": container with ID starting with 2a00cdcb81679a0aee22ff53dc9fd518122bc4d53b75eb7606debfbf3232a36d not found: ID does not exist" Jul 09 16:09:50.897896 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.897860 2577 scope.go:117] "RemoveContainer" containerID="5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e" Jul 09 16:09:50.898072 ip-10-0-133-0 kubenswrapper[2577]: E0709 16:09:50.898057 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e\": container with ID starting with 5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e not found: ID does not exist" containerID="5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e" Jul 09 16:09:50.898111 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.898074 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e"} err="failed to get container status \"5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e\": rpc error: code = NotFound desc = could not find container \"5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e\": container with ID starting with 5e85c3063553826f434a42ed0346f0872ad7a53bbaeaaa8136bb2fc30e420f7e not found: ID does not exist" Jul 09 16:09:50.903752 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.903730 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq"] Jul 09 16:09:50.910669 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:50.910649 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c8e6dd-openshift-default-7684f57f47-8dsnq"] Jul 09 16:09:52.136330 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:52.136298 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7167618b-bd7b-488d-b795-1306579a2420" path="/var/lib/kubelet/pods/7167618b-bd7b-488d-b795-1306579a2420/volumes" Jul 09 16:09:52.737279 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:52.737251 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb"] Jul 09 16:09:52.737475 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:52.737439 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" containerID="cri-o://eab809b5eb4721220946658f8e37137bc712c8ad1e34aaac67fd087e1adb2e5a" gracePeriod=30 Jul 09 16:09:55.089159 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:55.089124 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4jk65"] Jul 09 16:09:57.931170 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:57.931135 2577 generic.go:356] "Generic (PLEG): container finished" podID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerID="eab809b5eb4721220946658f8e37137bc712c8ad1e34aaac67fd087e1adb2e5a" exitCode=0 Jul 09 16:09:57.931796 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:57.931206 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" event={"ID":"b20680a9-6af1-401a-83f4-b8bfb7db2b2a","Type":"ContainerDied","Data":"eab809b5eb4721220946658f8e37137bc712c8ad1e34aaac67fd087e1adb2e5a"} Jul 09 16:09:57.931796 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:57.931267 2577 scope.go:117] "RemoveContainer" containerID="3bbf6d2c5915ea4909d5bfe00b931748e4f7f67d6d143e8f099147344cf9206a" Jul 09 16:09:57.993709 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:57.993681 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:09:58.080531 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080436 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-credential-socket\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080561 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-certs\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080656 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-podinfo\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080684 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp92s\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-kube-api-access-cp92s\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080874 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080713 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-token\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080874 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080761 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-envoy\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080874 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080786 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istiod-ca-cert\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.080874 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080792 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:58.080874 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080822 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-socket\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.081116 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.080881 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-data\") pod \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\" (UID: \"b20680a9-6af1-401a-83f4-b8bfb7db2b2a\") " Jul 09 16:09:58.081116 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.081021 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:58.081218 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.081140 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 16:09:58.081321 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.081305 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-certs\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.081321 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.081307 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:58.081415 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.081326 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istiod-ca-cert\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.081415 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.081341 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-workload-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.082106 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.082071 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:58.083002 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.082977 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-kube-api-access-cp92s" (OuterVolumeSpecName: "kube-api-access-cp92s") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "kube-api-access-cp92s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:09:58.083122 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.083062 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 16:09:58.083122 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.083073 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 16:09:58.083548 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.083521 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b20680a9-6af1-401a-83f4-b8bfb7db2b2a" (UID: "b20680a9-6af1-401a-83f4-b8bfb7db2b2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 16:09:58.183103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.183081 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-data\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.183103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.183104 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-credential-socket\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.183262 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.183117 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-podinfo\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.183262 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.183130 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp92s\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-kube-api-access-cp92s\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.183262 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.183141 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-token\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.183262 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.183152 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b20680a9-6af1-401a-83f4-b8bfb7db2b2a-istio-envoy\") on node \"ip-10-0-133-0.ec2.internal\" DevicePath \"\"" Jul 09 16:09:58.454700 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.454675 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xwc9s_cbc21ba3-7820-4cb8-b921-cb9f8072497a/manager/0.log" Jul 09 16:09:58.587035 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.587004 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ghsxn_9dffc73f-6771-46be-a7ba-ab6b22d45cd5/manager/0.log" Jul 09 16:09:58.938602 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.938574 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" Jul 09 16:09:58.938602 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.938589 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb" event={"ID":"b20680a9-6af1-401a-83f4-b8bfb7db2b2a","Type":"ContainerDied","Data":"796f25c401945c6a4d2502bcdc617e350f74831b45171aed05df1f71053c4753"} Jul 09 16:09:58.939035 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.938648 2577 scope.go:117] "RemoveContainer" containerID="eab809b5eb4721220946658f8e37137bc712c8ad1e34aaac67fd087e1adb2e5a" Jul 09 16:09:58.959054 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.959028 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb"] Jul 09 16:09:58.963446 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.963422 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1a4b78-openshift-default-6bbc4dff5-hptgb"] Jul 09 16:09:58.984891 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:58.984868 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r9mvq_27a197e8-3ab4-428d-9a43-9672a5087c60/manager/0.log" Jul 09 16:09:59.401531 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:59.401506 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bb557cf6-zrptf_814184a8-c96c-4647-a69a-d9355265e3f4/copy-manifests/0.log" Jul 09 16:09:59.413925 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:09:59.413888 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bb557cf6-zrptf_814184a8-c96c-4647-a69a-d9355265e3f4/manager/0.log" Jul 09 16:10:00.136592 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.136562 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" path="/var/lib/kubelet/pods/b20680a9-6af1-401a-83f4-b8bfb7db2b2a/volumes" Jul 09 16:10:00.258657 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.258630 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85_79ba2662-090a-41a1-a2d6-a79ff5d2cf32/util/0.log" Jul 09 16:10:00.267631 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.267610 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85_79ba2662-090a-41a1-a2d6-a79ff5d2cf32/pull/0.log" Jul 09 16:10:00.276614 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.276591 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85_79ba2662-090a-41a1-a2d6-a79ff5d2cf32/extract/0.log" Jul 09 16:10:00.394763 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.394698 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg_db69eeea-a3d6-42d4-8bf5-c6374630872e/extract/0.log" Jul 09 16:10:00.402927 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.402903 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg_db69eeea-a3d6-42d4-8bf5-c6374630872e/util/0.log" Jul 09 16:10:00.415539 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.415520 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg_db69eeea-a3d6-42d4-8bf5-c6374630872e/pull/0.log" Jul 09 16:10:00.537359 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.537331 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb_ebf730d1-ba6b-4e63-8208-4f81502ee664/util/0.log" Jul 09 16:10:00.546170 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.546152 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb_ebf730d1-ba6b-4e63-8208-4f81502ee664/pull/0.log" Jul 09 16:10:00.554167 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.554146 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb_ebf730d1-ba6b-4e63-8208-4f81502ee664/extract/0.log" Jul 09 16:10:00.671969 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.671900 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk_83216ab3-6a0c-44ae-a3c1-8806d1be5f41/pull/0.log" Jul 09 16:10:00.679434 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.679414 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk_83216ab3-6a0c-44ae-a3c1-8806d1be5f41/extract/0.log" Jul 09 16:10:00.686160 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.686143 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk_83216ab3-6a0c-44ae-a3c1-8806d1be5f41/util/0.log" Jul 09 16:10:00.943025 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:00.942948 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-sb8wz_62429a0e-7f38-4b9d-b492-bc4a6210d4c9/manager/0.log" Jul 09 16:10:01.063432 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:01.063400 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cf845_9a958d42-ab21-4473-891e-c48491d9abc5/manager/0.log" Jul 09 16:10:01.184339 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:01.184310 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-58t2k_98c78943-6895-4e9f-8a49-a86c70fbb27f/kuadrant-console-plugin/0.log" Jul 09 16:10:01.303083 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:01.303015 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dwjs4_e1309ef6-aada-4d11-a641-8e510e7f5c50/registry-server/0.log" Jul 09 16:10:01.431724 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:01.431700 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vnbh5_e6de3174-5ce8-4b25-ac0f-cda8cb199875/manager/0.log" Jul 09 16:10:01.560396 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:01.560327 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4jk65_aca775a8-1680-4ddf-96c2-873210eb6ccd/limitador/0.log" Jul 09 16:10:01.684909 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:01.684886 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fhcsc_425a9a1a-294c-4bd2-90ed-2753e0f5114e/manager/0.log" Jul 09 16:10:02.444541 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:02.444515 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f9565468-khrgp_95b0aa18-600f-4596-a181-670b4d1d3c02/kube-auth-proxy/0.log" Jul 09 16:10:02.570435 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:02.570409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/1.log" Jul 09 16:10:02.949729 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:02.949704 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dc6bcc468-7nttr_08707e88-88a7-46f7-b9c7-80008e16b603/router/0.log" Jul 09 16:10:03.565481 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:03.565440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-92gnf_692268f1-7155-4d38-8c4b-bfd199684495/main/0.log" Jul 09 16:10:03.921958 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:03.921935 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-hv7cs_44a46cf9-215e-40e6-83c7-b76d0a99fb05/main/0.log" Jul 09 16:10:08.347379 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347345 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9qrm8/must-gather-jjvwx"] Jul 09 16:10:08.347769 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347749 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" Jul 09 16:10:08.347769 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347761 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" Jul 09 16:10:08.347838 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347775 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" Jul 09 16:10:08.347838 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347781 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" Jul 09 16:10:08.347838 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347788 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" Jul 09 16:10:08.347838 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347795 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" Jul 09 16:10:08.347958 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347863 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" Jul 09 16:10:08.347958 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347872 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" Jul 09 16:10:08.347958 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347877 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" Jul 09 16:10:08.347958 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347884 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7167618b-bd7b-488d-b795-1306579a2420" containerName="istio-proxy" Jul 09 16:10:08.348083 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347959 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" Jul 09 16:10:08.348083 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.347964 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20680a9-6af1-401a-83f4-b8bfb7db2b2a" containerName="istio-proxy" Jul 09 16:10:08.351257 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.351242 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.353381 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.353361 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9qrm8\"/\"openshift-service-ca.crt\"" Jul 09 16:10:08.353511 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.353389 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9qrm8\"/\"kube-root-ca.crt\"" Jul 09 16:10:08.353805 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.353793 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9qrm8\"/\"default-dockercfg-xvqd8\"" Jul 09 16:10:08.359051 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.358911 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9qrm8/must-gather-jjvwx"] Jul 09 16:10:08.470151 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.470120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/95e35ea5-67c8-4575-bf32-06fcf410ee20-must-gather-output\") pod \"must-gather-jjvwx\" (UID: \"95e35ea5-67c8-4575-bf32-06fcf410ee20\") " pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.470292 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.470192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnrm\" (UniqueName: \"kubernetes.io/projected/95e35ea5-67c8-4575-bf32-06fcf410ee20-kube-api-access-fmnrm\") pod \"must-gather-jjvwx\" (UID: \"95e35ea5-67c8-4575-bf32-06fcf410ee20\") " pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.570893 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.570865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/95e35ea5-67c8-4575-bf32-06fcf410ee20-must-gather-output\") pod \"must-gather-jjvwx\" (UID: \"95e35ea5-67c8-4575-bf32-06fcf410ee20\") " pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.570996 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.570916 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnrm\" (UniqueName: \"kubernetes.io/projected/95e35ea5-67c8-4575-bf32-06fcf410ee20-kube-api-access-fmnrm\") pod \"must-gather-jjvwx\" (UID: \"95e35ea5-67c8-4575-bf32-06fcf410ee20\") " pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.571174 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.571156 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/95e35ea5-67c8-4575-bf32-06fcf410ee20-must-gather-output\") pod \"must-gather-jjvwx\" (UID: \"95e35ea5-67c8-4575-bf32-06fcf410ee20\") " pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.585851 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.585822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnrm\" (UniqueName: \"kubernetes.io/projected/95e35ea5-67c8-4575-bf32-06fcf410ee20-kube-api-access-fmnrm\") pod \"must-gather-jjvwx\" (UID: \"95e35ea5-67c8-4575-bf32-06fcf410ee20\") " pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.661834 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.661816 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9qrm8/must-gather-jjvwx" Jul 09 16:10:08.796757 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.796735 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9qrm8/must-gather-jjvwx"] Jul 09 16:10:08.798774 ip-10-0-133-0 kubenswrapper[2577]: W0709 16:10:08.798748 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95e35ea5_67c8_4575_bf32_06fcf410ee20.slice/crio-6df82b9a43be7d289bf0ae323a0baff7c3a446790784ba10f8b67db45e349e02 WatchSource:0}: Error finding container 6df82b9a43be7d289bf0ae323a0baff7c3a446790784ba10f8b67db45e349e02: Status 404 returned error can't find the container with id 6df82b9a43be7d289bf0ae323a0baff7c3a446790784ba10f8b67db45e349e02 Jul 09 16:10:08.800517 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.800496 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 16:10:08.997368 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:08.997285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9qrm8/must-gather-jjvwx" event={"ID":"95e35ea5-67c8-4575-bf32-06fcf410ee20","Type":"ContainerStarted","Data":"6df82b9a43be7d289bf0ae323a0baff7c3a446790784ba10f8b67db45e349e02"} Jul 09 16:10:12.017360 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:12.017327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9qrm8/must-gather-jjvwx" event={"ID":"95e35ea5-67c8-4575-bf32-06fcf410ee20","Type":"ContainerStarted","Data":"ffb335435107710102087a68106599178b5d499d8e193f78b694c9a8bd38e438"} Jul 09 16:10:12.017790 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:12.017369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9qrm8/must-gather-jjvwx" event={"ID":"95e35ea5-67c8-4575-bf32-06fcf410ee20","Type":"ContainerStarted","Data":"8a443053d395741a27b1056d68cf9e4cc5c69b7d9223241047dbce2365be302e"} Jul 09 16:10:12.036070 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:12.036009 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9qrm8/must-gather-jjvwx" podStartSLOduration=1.641144669 podStartE2EDuration="4.035990623s" totalImagesPullingTime="2.394845954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:10:12.033576507 +0000 UTC m=+2328.570803680" watchObservedRunningTime="2026-07-09 16:10:12.035990623 +0000 UTC m=+2328.573217798" Jul 09 16:10:12.759568 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:12.759523 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8nz45_b9b7bb8a-a0ca-4433-aa81-bf03c30c5a6d/global-pull-secret-syncer/0.log" Jul 09 16:10:12.924913 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:12.924883 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9m4mg_5f176317-d27b-4f21-b235-359b94766bbd/konnectivity-agent/0.log" Jul 09 16:10:13.007546 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:13.007519 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-0.ec2.internal_0df129d73aab3b87bad4f6dc5940ca16/haproxy/0.log" Jul 09 16:10:16.987324 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:16.987287 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85_79ba2662-090a-41a1-a2d6-a79ff5d2cf32/extract/0.log" Jul 09 16:10:17.013575 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.013547 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85_79ba2662-090a-41a1-a2d6-a79ff5d2cf32/util/0.log" Jul 09 16:10:17.046389 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.046328 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m8s85_79ba2662-090a-41a1-a2d6-a79ff5d2cf32/pull/0.log" Jul 09 16:10:17.078590 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.078536 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg_db69eeea-a3d6-42d4-8bf5-c6374630872e/extract/0.log" Jul 09 16:10:17.105135 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.105108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg_db69eeea-a3d6-42d4-8bf5-c6374630872e/util/0.log" Jul 09 16:10:17.134769 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.134729 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lvmgg_db69eeea-a3d6-42d4-8bf5-c6374630872e/pull/0.log" Jul 09 16:10:17.165940 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.165903 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb_ebf730d1-ba6b-4e63-8208-4f81502ee664/extract/0.log" Jul 09 16:10:17.190237 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.190094 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb_ebf730d1-ba6b-4e63-8208-4f81502ee664/util/0.log" Jul 09 16:10:17.216092 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.215894 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c44xb_ebf730d1-ba6b-4e63-8208-4f81502ee664/pull/0.log" Jul 09 16:10:17.252347 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.252238 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk_83216ab3-6a0c-44ae-a3c1-8806d1be5f41/extract/0.log" Jul 09 16:10:17.276922 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.276892 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk_83216ab3-6a0c-44ae-a3c1-8806d1be5f41/util/0.log" Jul 09 16:10:17.303852 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.303792 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r4bzk_83216ab3-6a0c-44ae-a3c1-8806d1be5f41/pull/0.log" Jul 09 16:10:17.605321 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.605144 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-sb8wz_62429a0e-7f38-4b9d-b492-bc4a6210d4c9/manager/0.log" Jul 09 16:10:17.632649 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.632429 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cf845_9a958d42-ab21-4473-891e-c48491d9abc5/manager/0.log" Jul 09 16:10:17.673535 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.673500 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-58t2k_98c78943-6895-4e9f-8a49-a86c70fbb27f/kuadrant-console-plugin/0.log" Jul 09 16:10:17.718350 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.718173 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dwjs4_e1309ef6-aada-4d11-a641-8e510e7f5c50/registry-server/0.log" Jul 09 16:10:17.885267 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.885230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vnbh5_e6de3174-5ce8-4b25-ac0f-cda8cb199875/manager/0.log" Jul 09 16:10:17.914780 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.914757 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4jk65_aca775a8-1680-4ddf-96c2-873210eb6ccd/limitador/0.log" Jul 09 16:10:17.983604 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:17.983574 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fhcsc_425a9a1a-294c-4bd2-90ed-2753e0f5114e/manager/0.log" Jul 09 16:10:19.959550 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:19.959425 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-s5fns_55fb2e48-ed73-44be-bf45-c15342026d24/cluster-monitoring-operator/0.log" Jul 09 16:10:19.996184 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:19.996110 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wmvr9_12199445-7494-49e4-aa80-3f6f435007dd/kube-state-metrics/0.log" Jul 09 16:10:20.018079 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.018044 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wmvr9_12199445-7494-49e4-aa80-3f6f435007dd/kube-rbac-proxy-main/0.log" Jul 09 16:10:20.049387 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.049355 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wmvr9_12199445-7494-49e4-aa80-3f6f435007dd/kube-rbac-proxy-self/0.log" Jul 09 16:10:20.239194 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.239087 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5m6f_e71e8e47-9ac8-4a5b-af1e-f1182509f332/node-exporter/0.log" Jul 09 16:10:20.264334 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.264292 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5m6f_e71e8e47-9ac8-4a5b-af1e-f1182509f332/kube-rbac-proxy/0.log" Jul 09 16:10:20.292277 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.292250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5m6f_e71e8e47-9ac8-4a5b-af1e-f1182509f332/init-textfile/0.log" Jul 09 16:10:20.780216 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.780105 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-jqk9c_5c381064-16b8-45e0-b61e-b427bc74085f/prometheus-operator-admission-webhook/0.log" Jul 09 16:10:20.892321 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.892290 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bbd9b4c9-gfn4l_59ffaac6-e6df-43e4-b257-41d7f26c4312/thanos-query/0.log" Jul 09 16:10:20.917682 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.917648 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bbd9b4c9-gfn4l_59ffaac6-e6df-43e4-b257-41d7f26c4312/kube-rbac-proxy-web/0.log" Jul 09 16:10:20.944500 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.944475 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bbd9b4c9-gfn4l_59ffaac6-e6df-43e4-b257-41d7f26c4312/kube-rbac-proxy/0.log" Jul 09 16:10:20.972973 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.972946 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bbd9b4c9-gfn4l_59ffaac6-e6df-43e4-b257-41d7f26c4312/prom-label-proxy/0.log" Jul 09 16:10:20.998262 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:20.998237 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bbd9b4c9-gfn4l_59ffaac6-e6df-43e4-b257-41d7f26c4312/kube-rbac-proxy-rules/0.log" Jul 09 16:10:21.022643 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:21.022619 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bbd9b4c9-gfn4l_59ffaac6-e6df-43e4-b257-41d7f26c4312/kube-rbac-proxy-metrics/0.log" Jul 09 16:10:22.235722 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.235687 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg"] Jul 09 16:10:22.243103 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.243077 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.246552 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.246525 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg"] Jul 09 16:10:22.432492 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.432440 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-sys\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.432661 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.432506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l64xm\" (UniqueName: \"kubernetes.io/projected/21ba8a7b-34b1-4305-bd51-69756aafbd5c-kube-api-access-l64xm\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.432661 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.432591 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-proc\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.432661 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.432631 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-lib-modules\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.432807 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.432710 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-podres\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533336 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-lib-modules\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533522 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-podres\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533522 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-sys\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533522 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xm\" (UniqueName: \"kubernetes.io/projected/21ba8a7b-34b1-4305-bd51-69756aafbd5c-kube-api-access-l64xm\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533522 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533427 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-lib-modules\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533522 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533510 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-sys\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533830 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533553 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-podres\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533830 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-proc\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.533938 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.533840 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21ba8a7b-34b1-4305-bd51-69756aafbd5c-proc\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.543818 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.543786 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l64xm\" (UniqueName: \"kubernetes.io/projected/21ba8a7b-34b1-4305-bd51-69756aafbd5c-kube-api-access-l64xm\") pod \"perf-node-gather-daemonset-4nlgg\" (UID: \"21ba8a7b-34b1-4305-bd51-69756aafbd5c\") " pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.555248 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.555219 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:22.752576 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:22.750962 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg"] Jul 09 16:10:23.112434 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.112348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" event={"ID":"21ba8a7b-34b1-4305-bd51-69756aafbd5c","Type":"ContainerStarted","Data":"c8e280b3fe12e25eeda2c05b300fc4799d47ca9de4a35f700cc923babcfb8edb"} Jul 09 16:10:23.112434 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.112406 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" event={"ID":"21ba8a7b-34b1-4305-bd51-69756aafbd5c","Type":"ContainerStarted","Data":"507a093fb538b009dbc7de70b5d97c98339bd178d907748ca9081e645a077fa2"} Jul 09 16:10:23.112633 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.112550 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:23.130763 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.130721 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" podStartSLOduration=1.130707552 podStartE2EDuration="1.130707552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:10:23.129100304 +0000 UTC m=+2339.666327476" watchObservedRunningTime="2026-07-09 16:10:23.130707552 +0000 UTC m=+2339.667934724" Jul 09 16:10:23.383136 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.383097 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d486fd6d8-fxfb6_796ee29c-723a-482e-865c-003658434a50/console/0.log" Jul 09 16:10:23.423351 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.423314 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-w2vmq_25ee3b41-af6d-4d49-a813-c9eee12fc550/download-server/0.log" Jul 09 16:10:23.998510 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:23.998481 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-nszjm_ea683fb5-2da7-4b3c-a8d9-42cfbfd03040/volume-data-source-validator/0.log" Jul 09 16:10:25.177227 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:25.177193 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvpzg_f46785e3-38b5-4c15-8891-270e1983b38e/dns/0.log" Jul 09 16:10:25.203479 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:25.203448 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvpzg_f46785e3-38b5-4c15-8891-270e1983b38e/kube-rbac-proxy/0.log" Jul 09 16:10:25.234651 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:25.234621 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4zmmk_45f9fd98-9365-48d7-a594-1ca287fd2234/dns-node-resolver/0.log" Jul 09 16:10:25.858893 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:25.858867 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d9c667bf6-bfgbb_78b56a34-ea76-4944-99b5-11b561194852/registry/0.log" Jul 09 16:10:25.883027 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:25.882994 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cz222_e873854a-76a3-4172-ae5e-bbd3af1a04b7/node-ca/0.log" Jul 09 16:10:27.082345 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:27.082318 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f9565468-khrgp_95b0aa18-600f-4596-a181-670b4d1d3c02/kube-auth-proxy/0.log" Jul 09 16:10:27.114889 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:27.114858 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/0.log" Jul 09 16:10:27.150870 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:27.150845 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/1.log" Jul 09 16:10:27.260928 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:27.260898 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dc6bcc468-7nttr_08707e88-88a7-46f7-b9c7-80008e16b603/router/0.log" Jul 09 16:10:27.939116 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:27.939094 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5lgqr_200101ee-c3b4-49b8-93da-a6cf99fa6061/serve-healthcheck-canary/0.log" Jul 09 16:10:28.468054 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:28.467949 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-9xxq9_7c955eea-454f-483b-b3ab-4c96b2d4d8cd/insights-operator/0.log" Jul 09 16:10:28.644921 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:28.644886 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mw449_2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8/kube-rbac-proxy/0.log" Jul 09 16:10:28.670409 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:28.670385 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mw449_2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8/exporter/0.log" Jul 09 16:10:28.700951 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:28.700928 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mw449_2b8ee5ad-e3a6-4b98-a568-fdcf98b0c4f8/extractor/0.log" Jul 09 16:10:29.132596 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:29.132560 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9qrm8/perf-node-gather-daemonset-4nlgg" Jul 09 16:10:30.952865 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:30.952842 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xwc9s_cbc21ba3-7820-4cb8-b921-cb9f8072497a/manager/0.log" Jul 09 16:10:31.101354 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:31.101278 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ghsxn_9dffc73f-6771-46be-a7ba-ab6b22d45cd5/manager/0.log" Jul 09 16:10:31.298066 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:31.297388 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r9mvq_27a197e8-3ab4-428d-9a43-9672a5087c60/manager/0.log" Jul 09 16:10:31.475646 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:31.475618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bb557cf6-zrptf_814184a8-c96c-4647-a69a-d9355265e3f4/manager/0.log" Jul 09 16:10:31.503133 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:31.503108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bb557cf6-zrptf_814184a8-c96c-4647-a69a-d9355265e3f4/copy-manifests/0.log" Jul 09 16:10:32.273253 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:32.273228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f9565468-khrgp_95b0aa18-600f-4596-a181-670b4d1d3c02/kube-auth-proxy/0.log" Jul 09 16:10:32.294450 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:32.294428 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/0.log" Jul 09 16:10:32.331328 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:32.331290 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/1.log" Jul 09 16:10:32.411526 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:32.411497 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dc6bcc468-7nttr_08707e88-88a7-46f7-b9c7-80008e16b603/router/0.log" Jul 09 16:10:33.700338 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:33.700307 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f9565468-khrgp_95b0aa18-600f-4596-a181-670b4d1d3c02/kube-auth-proxy/0.log" Jul 09 16:10:33.725896 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:33.725855 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/0.log" Jul 09 16:10:33.768379 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:33.768320 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gphgg_976f3760-fff0-44bc-821f-3d206f41251a/istio-proxy/1.log" Jul 09 16:10:33.857778 ip-10-0-133-0 kubenswrapper[2577]: I0709 16:10:33.857748 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dc6bcc468-7nttr_08707e88-88a7-46f7-b9c7-80008e16b603/router/0.log"