Jul 07 21:58:41.285629 ip-10-0-132-125 systemd[1]: Starting Kubernetes Kubelet... Jul 07 21:58:41.725788 ip-10-0-132-125 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 21:58:41.725788 ip-10-0-132-125 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 21:58:41.725788 ip-10-0-132-125 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 21:58:41.725788 ip-10-0-132-125 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 21:58:41.725788 ip-10-0-132-125 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 21:58:41.726511 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.725801 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 21:58:41.732262 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732239 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 21:58:41.732262 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732261 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732265 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732269 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732273 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732277 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732280 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732283 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732286 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732288 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732292 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732294 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732297 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732300 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732302 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732305 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732307 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732310 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732312 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732315 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732323 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 21:58:41.732357 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732339 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732341 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732344 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732347 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732350 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732352 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732355 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732358 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732360 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732363 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732366 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732369 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732371 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732374 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732376 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732379 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732381 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732384 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732388 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 21:58:41.732827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732392 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732395 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732397 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732400 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732403 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732406 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732408 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732411 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732414 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732417 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732420 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732422 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732425 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732430 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732432 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732436 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732438 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732441 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732444 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732447 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 21:58:41.733289 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732450 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732453 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732455 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732458 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732461 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732463 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732466 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732468 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732471 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732473 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732476 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732478 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732480 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732483 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732485 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732488 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732490 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732493 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732495 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732498 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 21:58:41.733781 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732500 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732503 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732505 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732508 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732510 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732515 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732519 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732523 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732526 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732528 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732531 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732533 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732536 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732539 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732541 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732544 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732546 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732551 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.732555 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 21:58:41.734387 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733280 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733287 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733290 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733293 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733297 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733300 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733303 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733306 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733309 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733312 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733315 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733317 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733320 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733322 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733340 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733343 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733346 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733348 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733351 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733353 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733358 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 21:58:41.734842 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733361 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733363 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733366 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733368 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733371 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733373 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733376 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733378 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733381 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733383 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733385 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733388 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733390 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733393 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733395 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733398 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733401 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733404 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733407 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733409 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 21:58:41.735385 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733412 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733414 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733416 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733419 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733421 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733423 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733426 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733428 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733431 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733433 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733436 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733439 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733442 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733446 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733450 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733454 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733456 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733458 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733461 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 21:58:41.735854 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733463 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733466 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733468 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733471 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733474 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733476 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733479 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733481 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733484 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733486 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733489 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733492 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733494 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733496 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733499 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733501 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733504 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733506 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733508 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733511 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 21:58:41.736324 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733513 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733516 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733520 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733524 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733527 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733530 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733533 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733536 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733539 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733543 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733546 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733548 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733551 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733554 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733556 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733559 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733561 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733564 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733566 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 21:58:41.736827 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733976 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733982 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733985 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733987 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733990 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733993 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733996 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.733998 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734002 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734006 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734009 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734012 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734014 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734017 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734020 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734022 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734025 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734028 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734031 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 21:58:41.737304 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734038 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734042 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734045 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734047 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734050 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734054 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734057 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734060 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734062 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734065 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734067 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734070 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734072 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734075 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734077 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734080 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734083 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734085 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734088 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734091 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 21:58:41.737761 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734093 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734096 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734098 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734100 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734104 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734106 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734109 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734111 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734114 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734123 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734125 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734128 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734130 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734134 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734136 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734139 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734142 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734144 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734147 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734149 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 21:58:41.738248 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734152 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734154 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734157 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734159 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734161 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734164 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734166 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734169 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734171 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734173 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734176 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734178 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734181 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734183 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734185 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734187 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734190 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734193 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734196 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 21:58:41.738782 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734199 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734203 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734207 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734210 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734213 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734215 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734218 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734221 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734224 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734226 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734228 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734231 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734233 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734236 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734238 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734241 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734243 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734245 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734248 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 21:58:41.739238 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734250 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734252 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734346 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734365 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734372 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734376 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734383 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734386 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734391 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734395 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734399 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734402 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734405 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734408 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734411 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734415 2562 flags.go:64] FLAG: --cgroup-root="" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734418 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734421 2562 flags.go:64] FLAG: --client-ca-file="" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734424 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734427 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734431 2562 flags.go:64] FLAG: --cluster-domain="" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734434 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734437 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734440 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 21:58:41.739734 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734444 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734448 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734450 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734453 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734457 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734459 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734462 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734466 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734469 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734471 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734478 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734481 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734484 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734487 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734490 2562 flags.go:64] FLAG: --enable-server="true" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734493 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734498 2562 flags.go:64] FLAG: --event-burst="100" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734500 2562 flags.go:64] FLAG: --event-qps="50" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734503 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734506 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734509 2562 flags.go:64] FLAG: --eviction-hard="" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734512 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734515 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734519 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734522 2562 flags.go:64] FLAG: --eviction-soft="" Jul 07 21:58:41.740310 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734524 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734527 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734530 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734533 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734536 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734539 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734541 2562 flags.go:64] FLAG: --feature-gates="" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734545 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734548 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734551 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734554 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734557 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734559 2562 flags.go:64] FLAG: --help="false" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734562 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-125.ec2.internal" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734565 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734568 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734571 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734575 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734578 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734581 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734583 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734586 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734589 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 21:58:41.740920 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734592 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734595 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734597 2562 flags.go:64] FLAG: --kube-reserved="" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734600 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734603 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734606 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734608 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734612 2562 flags.go:64] FLAG: --lock-file="" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734615 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734618 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734621 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734626 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734629 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734632 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734634 2562 flags.go:64] FLAG: --logging-format="text" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734637 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734641 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734644 2562 flags.go:64] FLAG: --manifest-url="" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734646 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734651 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734653 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734657 2562 flags.go:64] FLAG: --max-pods="110" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734660 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734663 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734666 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 21:58:41.741502 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734668 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734673 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734676 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734679 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734686 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734689 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734692 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734695 2562 flags.go:64] FLAG: --pod-cidr="" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734698 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734704 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734707 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734710 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734712 2562 flags.go:64] FLAG: --port="10250" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734715 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734720 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05de08add35e2d8a8" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734723 2562 flags.go:64] FLAG: --qos-reserved="" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734725 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734728 2562 flags.go:64] FLAG: --register-node="true" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734731 2562 flags.go:64] FLAG: --register-with-taints="" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734735 2562 flags.go:64] FLAG: --registry-burst="10" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734738 2562 flags.go:64] FLAG: --registry-qps="5" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734740 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734743 2562 flags.go:64] FLAG: --reserved-memory="" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734747 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734751 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 21:58:41.742405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734754 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734756 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734759 2562 flags.go:64] FLAG: --runonce="false" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734762 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734765 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734768 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734771 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734773 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734776 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734780 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734783 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734786 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734788 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734791 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734794 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734797 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734800 2562 flags.go:64] FLAG: --system-cgroups="" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734803 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734809 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734811 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734814 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734819 2562 flags.go:64] FLAG: --tls-min-version="" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734822 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734825 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734828 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 21:58:41.743200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734831 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734833 2562 flags.go:64] FLAG: --v="2" Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734837 2562 flags.go:64] FLAG: --version="false" Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734841 2562 flags.go:64] FLAG: --vmodule="" Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734845 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.734848 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734968 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734971 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734974 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734977 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734981 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734984 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734988 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734991 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734994 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.734996 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735000 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735003 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735006 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735009 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735011 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 21:58:41.743866 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735014 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735016 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735019 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735021 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735024 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735026 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735029 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735033 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735035 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735037 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735040 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735042 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735045 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735047 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735050 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735053 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735055 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735058 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735060 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735063 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735066 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 21:58:41.744395 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735068 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735070 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735072 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735075 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735077 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735080 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735083 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735086 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735088 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735091 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735093 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735097 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735100 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735103 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735106 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735109 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735111 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735114 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735118 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 21:58:41.744908 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735120 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735123 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735125 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735128 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735130 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735133 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735135 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735137 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735140 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735142 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735145 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735150 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735152 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735155 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735157 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735160 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735163 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735165 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735167 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 21:58:41.745386 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735171 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735173 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735176 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735179 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735181 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735184 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735186 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735189 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735191 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735194 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735196 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735199 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735202 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735205 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735207 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735209 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735212 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735214 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735218 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 21:58:41.745851 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735221 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735224 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735227 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735230 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735232 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.735235 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.735962 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.744107 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.744126 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.745088 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.745096 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745162 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745167 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745170 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745174 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 21:58:41.746302 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745177 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745180 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745183 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745186 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745188 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745191 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745193 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745196 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745198 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745201 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745204 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745207 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745210 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745212 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745215 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745217 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745220 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745222 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745225 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745230 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 21:58:41.746695 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745234 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745237 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745239 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745242 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745246 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745248 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745251 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745253 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745256 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745259 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745261 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745264 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745266 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745269 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745272 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745274 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745277 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745280 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745284 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 21:58:41.747190 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745287 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745290 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745293 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745295 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745298 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745300 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745303 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745305 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745308 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745310 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745313 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745315 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745319 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745321 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745324 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745337 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745340 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745342 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745345 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745347 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 21:58:41.747733 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745350 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745352 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745355 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745357 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745360 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745363 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745365 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745367 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745370 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745372 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745375 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745377 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745380 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745382 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745385 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745387 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745391 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745394 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745397 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745400 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 21:58:41.748219 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745403 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745405 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745408 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745410 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745413 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745415 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745418 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745420 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745423 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745425 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745428 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745430 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745433 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745436 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745440 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 21:58:41.748713 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745443 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.745449 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745566 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745571 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745574 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745578 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745580 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745583 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745586 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745589 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745591 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745594 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745597 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745599 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745604 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 21:58:41.749073 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745612 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745615 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745619 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745622 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745624 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745627 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745629 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745632 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745634 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745637 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745639 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745643 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745646 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745649 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745651 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745654 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745656 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745659 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745661 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 21:58:41.749498 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745664 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745667 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745669 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745672 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745674 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745677 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745680 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745682 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745685 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745688 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745690 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745693 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745695 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745699 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745701 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745703 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745706 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745709 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745712 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745715 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 21:58:41.749949 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745717 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745719 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745721 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745724 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745726 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745729 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745732 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745735 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745739 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745742 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745744 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745747 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745750 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745753 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745755 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745758 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745761 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745763 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745765 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745768 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 21:58:41.750445 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745770 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745775 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745777 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745780 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745782 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745784 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745787 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745789 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745792 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745795 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745797 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745799 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745802 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745804 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745806 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745809 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745811 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745813 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745816 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745818 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 21:58:41.750912 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745821 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745823 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745825 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745828 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745830 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745833 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:41.745836 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.745841 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.745965 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 21:58:41.751400 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.750087 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 21:58:41.751629 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.751464 2562 server.go:1019] "Starting client certificate rotation" Jul 07 21:58:41.751629 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.751576 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 21:58:41.751629 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.751614 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 21:58:41.754104 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.754089 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 21:58:41.755496 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.755481 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 21:58:41.764363 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.764342 2562 log.go:25] "Validated CRI v1 runtime API" Jul 07 21:58:41.768718 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.768698 2562 log.go:25] "Validated CRI v1 image API" Jul 07 21:58:41.769577 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.769561 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 21:58:41.772196 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.772172 2562 fs.go:135] Filesystem UUIDs: map[64426b0c-5dc3-43cc-89b5-4bf4e36144f6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7b4d6e8a-7de5-4e15-b8c4-9b567c617740:/dev/nvme0n1p4] Jul 07 21:58:41.772255 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.772195 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 21:58:41.779417 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.779283 2562 manager.go:216] Machine: {Timestamp:2026-07-07 21:58:41.77760484 +0000 UTC m=+0.372364000 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100814 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2194139ae8a80c235a28042a3555dd SystemUUID:ec219413-9ae8-a80c-235a-28042a3555dd BootID:f104ccf1-3e10-46fd-91fd-2639a00c9106 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:89:60:73:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:89:60:73:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:10:01:16:f3:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 21:58:41.779417 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.779408 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 21:58:41.779552 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.779494 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 21:58:41.783429 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.783397 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 21:58:41.783561 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.783432 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-125.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 21:58:41.784132 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.784121 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 21:58:41.784168 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.784133 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 21:58:41.784168 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.784141 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 21:58:41.784854 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.784844 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 21:58:41.786651 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.786640 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 21:58:41.787873 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.787856 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 21:58:41.789369 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789357 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 21:58:41.789489 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789480 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 21:58:41.789531 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789508 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 07 21:58:41.789531 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789519 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 21:58:41.789605 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789533 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 21:58:41.789605 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789542 2562 kubelet.go:398] "Adding apiserver pod source" Jul 07 21:58:41.789605 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789556 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 21:58:41.789927 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789916 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 21:58:41.789967 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.789945 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 21:58:41.792207 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.792189 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 21:58:41.794089 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.794075 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 21:58:41.794159 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.794106 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 21:58:41.794999 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.794987 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795004 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795010 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795015 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795025 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795030 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795038 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795045 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 21:58:41.795050 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795052 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 21:58:41.795264 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795058 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 21:58:41.795264 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795067 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 21:58:41.795264 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795074 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 21:58:41.795821 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795811 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 21:58:41.795821 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.795820 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 21:58:41.799272 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.799253 2562 server.go:1268] "Started kubelet" Jul 07 21:58:41.799399 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.799339 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 21:58:41.799568 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.799394 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 21:58:41.799670 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.799654 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 21:58:41.801124 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.801097 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 21:58:41.801214 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.801131 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-125.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 21:58:41.801376 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.801285 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 21:58:41.801499 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.801476 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 21:58:41.801585 ip-10-0-132-125 systemd[1]: Started Kubernetes Kubelet. Jul 07 21:58:41.802714 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.802699 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 07 21:58:41.808343 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.804372 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-125.ec2.internal.18c020ce0d6b8b5f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-125.ec2.internal,UID:ip-10-0-132-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-125.ec2.internal,},FirstTimestamp:2026-07-07 21:58:41.799220063 +0000 UTC m=+0.393979211,LastTimestamp:2026-07-07 21:58:41.799220063 +0000 UTC m=+0.393979211,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-125.ec2.internal,}" Jul 07 21:58:41.808343 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.805903 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 21:58:41.808625 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.808608 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 21:58:41.808673 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.808627 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 21:58:41.808777 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.808753 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:41.809102 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.809087 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 21:58:41.809102 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.809102 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 21:58:41.809238 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.809151 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 21:58:41.809872 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.809855 2562 factory.go:55] Registering systemd factory Jul 07 21:58:41.809872 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.809875 2562 factory.go:223] Registration of the systemd container factory successfully Jul 07 21:58:41.810125 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810113 2562 factory.go:153] Registering CRI-O factory Jul 07 21:58:41.810125 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810126 2562 factory.go:223] Registration of the crio container factory successfully Jul 07 21:58:41.810215 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810160 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 07 21:58:41.810215 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810171 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 07 21:58:41.810215 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810174 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 21:58:41.810215 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810201 2562 factory.go:103] Registering Raw factory Jul 07 21:58:41.810215 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810213 2562 manager.go:1196] Started watching for new ooms in manager Jul 07 21:58:41.810860 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.810844 2562 manager.go:349] Starting recovery of all containers Jul 07 21:58:41.821766 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.821571 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 21:58:41.821766 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.821572 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 21:58:41.824460 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.822962 2562 manager.go:354] Recovery completed Jul 07 21:58:41.824460 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.824278 2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 07 21:58:41.827543 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.827529 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:41.829736 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.829713 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:41.829823 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.829749 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:41.829823 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.829761 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:41.830921 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.830907 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 21:58:41.830921 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.830918 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 21:58:41.830921 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.830920 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:41.831079 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.830934 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 21:58:41.831933 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.831917 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:41.831993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.831944 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:41.831993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.831955 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:41.831993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.831957 2562 policy_none.go:49] "None policy: Start" Jul 07 21:58:41.831993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.831977 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 21:58:41.831993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.831991 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 21:58:41.832194 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.832096 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-125.ec2.internal.18c020ce0f3d2d5e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-125.ec2.internal,UID:ip-10-0-132-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-125.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-125.ec2.internal,},FirstTimestamp:2026-07-07 21:58:41.829735774 +0000 UTC m=+0.424494922,LastTimestamp:2026-07-07 21:58:41.829735774 +0000 UTC m=+0.424494922,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-125.ec2.internal,}" Jul 07 21:58:41.833199 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.833186 2562 policy_none.go:47] "Start" Jul 07 21:58:41.843193 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.843111 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-125.ec2.internal.18c020ce0f3d7839 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-125.ec2.internal,UID:ip-10-0-132-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-125.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-125.ec2.internal,},FirstTimestamp:2026-07-07 21:58:41.829754937 +0000 UTC m=+0.424514085,LastTimestamp:2026-07-07 21:58:41.829754937 +0000 UTC m=+0.424514085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-125.ec2.internal,}" Jul 07 21:58:41.843193 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.843192 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rcrfd" Jul 07 21:58:41.853603 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.853501 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-125.ec2.internal.18c020ce0f3d9fec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-125.ec2.internal,UID:ip-10-0-132-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-125.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-125.ec2.internal,},FirstTimestamp:2026-07-07 21:58:41.8297651 +0000 UTC m=+0.424524248,LastTimestamp:2026-07-07 21:58:41.8297651 +0000 UTC m=+0.424524248,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-125.ec2.internal,}" Jul 07 21:58:41.855221 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.855200 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rcrfd" Jul 07 21:58:41.868085 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868066 2562 manager.go:337] "Starting Device Plugin manager" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.868109 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868119 2562 server.go:87] "Starting device plugin registration server" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868440 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868451 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868529 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868612 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.868623 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.869225 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 21:58:41.879151 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.869266 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:41.946929 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.946877 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 21:58:41.948247 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.948215 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 21:58:41.948370 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.948269 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 21:58:41.948370 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.948300 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 21:58:41.948370 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.948360 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 21:58:41.951537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.951498 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 21:58:41.969605 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.969567 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:41.970683 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.970668 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:41.970760 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.970700 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:41.970760 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.970713 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:41.970760 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.970737 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-125.ec2.internal" Jul 07 21:58:41.980506 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:41.980429 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-125.ec2.internal" Jul 07 21:58:41.980506 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.980467 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-125.ec2.internal\": node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:41.999400 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:41.999366 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.049131 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.049096 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal"] Jul 07 21:58:42.049282 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.049184 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:42.050618 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.050594 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:42.050677 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.050625 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:42.050677 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.050635 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:42.051828 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.051816 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:42.051935 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.051919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.051978 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.051952 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:42.052564 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.052548 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:42.052647 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.052576 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:42.052647 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.052585 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:42.053014 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.052989 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:42.053014 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.053017 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:42.053200 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.053031 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:42.053596 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.053579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.053656 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.053625 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:42.054290 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.054274 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:42.054398 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.054303 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:42.054398 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.054316 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:42.081187 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.081163 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-125.ec2.internal\" not found" node="ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.085731 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.085713 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-125.ec2.internal\" not found" node="ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.099926 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.099905 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.111828 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.111800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebaa656986e8d73fd90f95493cd3249-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal\" (UID: \"3ebaa656986e8d73fd90f95493cd3249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.111928 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.111831 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74f5d17caa4e3653403abe6e004ae3b2-config\") pod \"kube-apiserver-proxy-ip-10-0-132-125.ec2.internal\" (UID: \"74f5d17caa4e3653403abe6e004ae3b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.111928 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.111850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ebaa656986e8d73fd90f95493cd3249-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal\" (UID: \"3ebaa656986e8d73fd90f95493cd3249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.200060 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.200013 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.222958 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.212406 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebaa656986e8d73fd90f95493cd3249-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal\" (UID: \"3ebaa656986e8d73fd90f95493cd3249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.222958 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.212449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74f5d17caa4e3653403abe6e004ae3b2-config\") pod \"kube-apiserver-proxy-ip-10-0-132-125.ec2.internal\" (UID: \"74f5d17caa4e3653403abe6e004ae3b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.222958 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.212466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ebaa656986e8d73fd90f95493cd3249-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal\" (UID: \"3ebaa656986e8d73fd90f95493cd3249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.222958 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.212508 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74f5d17caa4e3653403abe6e004ae3b2-config\") pod \"kube-apiserver-proxy-ip-10-0-132-125.ec2.internal\" (UID: \"74f5d17caa4e3653403abe6e004ae3b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.222958 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.212508 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebaa656986e8d73fd90f95493cd3249-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal\" (UID: \"3ebaa656986e8d73fd90f95493cd3249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.222958 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.212525 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ebaa656986e8d73fd90f95493cd3249-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal\" (UID: \"3ebaa656986e8d73fd90f95493cd3249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.300961 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.300872 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.384521 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.384487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.388345 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.388112 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:42.401567 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.401542 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.502622 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.502572 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.603124 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.603048 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.703734 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.703693 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.751210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.751180 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 21:58:42.751858 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.751378 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 21:58:42.770109 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.770083 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 21:58:42.788800 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.788776 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 21:58:42.804113 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.804088 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.856698 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.856623 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 21:53:41 +0000 UTC" deadline="2027-12-20 04:20:09.802704726 +0000 UTC" Jul 07 21:58:42.856698 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.856660 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12726h21m26.946048373s" Jul 07 21:58:42.882956 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.882922 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 21:58:42.884558 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.884535 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientMemory" Jul 07 21:58:42.884677 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.884571 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 21:58:42.884677 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.884582 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeHasSufficientPID" Jul 07 21:58:42.905083 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:42.905053 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:42.911926 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:42.911879 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebaa656986e8d73fd90f95493cd3249.slice/crio-d45b47ae904193ea05acfdd82f18791f085dc91bced752a61d1faea7cfa9634d WatchSource:0}: Error finding container d45b47ae904193ea05acfdd82f18791f085dc91bced752a61d1faea7cfa9634d: Status 404 returned error can't find the container with id d45b47ae904193ea05acfdd82f18791f085dc91bced752a61d1faea7cfa9634d Jul 07 21:58:42.912137 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:42.912115 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f5d17caa4e3653403abe6e004ae3b2.slice/crio-6df94783f47f3302ca6ee51f69a2916502a08016f4ae9a4cc5780e619aed9dfa WatchSource:0}: Error finding container 6df94783f47f3302ca6ee51f69a2916502a08016f4ae9a4cc5780e619aed9dfa: Status 404 returned error can't find the container with id 6df94783f47f3302ca6ee51f69a2916502a08016f4ae9a4cc5780e619aed9dfa Jul 07 21:58:42.916263 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.916244 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 21:58:42.952345 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.952271 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" event={"ID":"74f5d17caa4e3653403abe6e004ae3b2","Type":"ContainerStarted","Data":"6df94783f47f3302ca6ee51f69a2916502a08016f4ae9a4cc5780e619aed9dfa"} Jul 07 21:58:42.953359 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:42.953318 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" event={"ID":"3ebaa656986e8d73fd90f95493cd3249","Type":"ContainerStarted","Data":"d45b47ae904193ea05acfdd82f18791f085dc91bced752a61d1faea7cfa9634d"} Jul 07 21:58:43.005532 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.005494 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:43.106100 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.106058 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:43.206698 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.206614 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:43.307462 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.307432 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-125.ec2.internal\" not found" Jul 07 21:58:43.331161 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.331135 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 21:58:43.408779 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.408754 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" Jul 07 21:58:43.421449 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.421419 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 21:58:43.422453 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.422427 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" Jul 07 21:58:43.433996 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.433929 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 21:58:43.791082 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.791034 2562 apiserver.go:52] "Watching apiserver" Jul 07 21:58:43.798223 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.798197 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 21:58:43.798670 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.798636 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7fgv5","openshift-dns/node-resolver-vp854","openshift-image-registry/node-ca-m9qs8","openshift-network-diagnostics/network-check-target-cffq9","openshift-network-operator/iptables-alerter-265ks","openshift-ovn-kubernetes/ovnkube-node-69lbw","kube-system/global-pull-secret-syncer-l6chz","kube-system/konnectivity-agent-zfb55","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal","openshift-multus/multus-additional-cni-plugins-wc8mv","openshift-multus/multus-qxzqd","openshift-multus/network-metrics-daemon-htffx"] Jul 07 21:58:43.800950 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.800920 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.803031 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.803009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.804148 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.804130 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.804251 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.804179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 21:58:43.804306 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.804130 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.804939 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.804923 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 21:58:43.805031 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.804941 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rvnd6\"" Jul 07 21:58:43.805135 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.805123 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 21:58:43.805281 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.805269 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.806853 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.806823 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.806853 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.806843 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.807047 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.806877 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 21:58:43.807271 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.807253 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.807368 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.807280 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ff58k\"" Jul 07 21:58:43.807849 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.807830 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 21:58:43.807975 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.807954 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 21:58:43.809195 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.809170 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 21:58:43.809924 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.809699 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:43.809924 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.809787 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:43.811619 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.811598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:43.813626 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.813606 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:43.815987 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.815965 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.816076 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.816014 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pwzmm\"" Jul 07 21:58:43.816466 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.816444 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.816545 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.816466 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.816599 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.816591 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bs8lz\"" Jul 07 21:58:43.816880 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.816858 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.816955 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.816864 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 21:58:43.818628 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.818144 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:43.818628 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.818222 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:43.819169 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.819151 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 21:58:43.819402 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.819381 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.819930 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.819912 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.820109 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820094 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lqmfj\"" Jul 07 21:58:43.820438 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820413 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820597 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8zrx9\"" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820671 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-kubelet\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6dcab854-eee8-4d5d-a448-602298a4e71d-hosts-file\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-cni-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-systemd-units\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820964 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.820998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-k8s-cni-cncf-io\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-hostroot\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-conf-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/07efbfcf-2445-4754-93af-05c5041397b6-multus-daemon-config\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovnkube-config\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-os-release\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-cni-bin\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821148 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-etc-kubernetes\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d51c526d-3e94-4c65-821c-ae2b5466a4bb-serviceca\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.821234 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-systemd\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821208 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-cni-bin\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-729kl\" (UniqueName: \"kubernetes.io/projected/87f2a6f9-bda9-494a-beb3-ea35876802fa-kube-api-access-729kl\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-netns\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbx8h\" (UniqueName: \"kubernetes.io/projected/07efbfcf-2445-4754-93af-05c5041397b6-kube-api-access-gbx8h\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-kubelet\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-var-lib-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-node-log\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovn-node-metrics-cert\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-system-cni-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-cni-multus\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821496 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-etc-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-log-socket\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-cni-netd\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821561 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-cnibin\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.822100 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcghm\" (UniqueName: \"kubernetes.io/projected/6dcab854-eee8-4d5d-a448-602298a4e71d-kube-api-access-mcghm\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mndzw\" (UniqueName: \"kubernetes.io/projected/d51c526d-3e94-4c65-821c-ae2b5466a4bb-kube-api-access-mndzw\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-slash\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-run-netns\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-env-overrides\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-multus-certs\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcab854-eee8-4d5d-a448-602298a4e71d-tmp-dir\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d51c526d-3e94-4c65-821c-ae2b5466a4bb-host\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-ovn\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovnkube-script-lib\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07efbfcf-2445-4754-93af-05c5041397b6-cni-binary-copy\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.823030 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.821865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-socket-dir-parent\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.823592 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.823576 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.823657 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.823596 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.823841 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.823824 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.823993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.823881 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ntss6\"" Jul 07 21:58:43.825686 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.825669 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 21:58:43.826047 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.826028 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.826638 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.826504 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 21:58:43.826638 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.826556 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 21:58:43.826638 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.826587 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w2rqs\"" Jul 07 21:58:43.826863 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.826850 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 21:58:43.828152 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.828005 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 21:58:43.828152 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.828092 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:43.828152 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.828144 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 21:58:43.828362 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.828144 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78kmp\"" Jul 07 21:58:43.828362 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.828159 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:43.845357 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.845316 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-j755t" Jul 07 21:58:43.858362 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.858340 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-j755t" Jul 07 21:58:43.910373 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.910346 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 21:58:43.923065 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovn-node-metrics-cert\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-system-cni-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.923229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-cni-multus\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.923229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ng42\" (UniqueName: \"kubernetes.io/projected/a61b1873-a762-4e95-aa01-818198e15cc3-kube-api-access-8ng42\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:43.923229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-os-release\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.923229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-log-socket\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-cni-netd\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-cnibin\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcghm\" (UniqueName: \"kubernetes.io/projected/6dcab854-eee8-4d5d-a448-602298a4e71d-kube-api-access-mcghm\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mndzw\" (UniqueName: \"kubernetes.io/projected/d51c526d-3e94-4c65-821c-ae2b5466a4bb-kube-api-access-mndzw\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923369 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-device-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923420 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-cni-netd\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-cnibin\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923469 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-log-socket\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923473 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-system-cni-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-cni-multus\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.923557 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-run-netns\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcab854-eee8-4d5d-a448-602298a4e71d-tmp-dir\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d51c526d-3e94-4c65-821c-ae2b5466a4bb-host\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysconfig\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923875 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-run-netns\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923884 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysctl-conf\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923938 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 21:58:43.923959 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.923984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-kubelet\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924028 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysctl-d\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d51c526d-3e94-4c65-821c-ae2b5466a4bb-serviceca\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-modprobe-d\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-systemd-units\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-k8s-cni-cncf-io\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-hostroot\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-conf-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/07efbfcf-2445-4754-93af-05c5041397b6-multus-daemon-config\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924276 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-lib-modules\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924293 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d51c526d-3e94-4c65-821c-ae2b5466a4bb-host\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-os-release\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-kubelet\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924465 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-os-release\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924483 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-conf-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924530 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-systemd-units\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-etc-kubernetes\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-systemd\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.924705 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-kubelet-config\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924849 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d51c526d-3e94-4c65-821c-ae2b5466a4bb-serviceca\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924858 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcab854-eee8-4d5d-a448-602298a4e71d-tmp-dir\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924929 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-hostroot\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-cni-bin\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.924995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-k8s-cni-cncf-io\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.925052 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13e9d2e6-45f5-4e22-99b9-c8ce050e23b0-konnectivity-ca\") pod \"konnectivity-agent-zfb55\" (UID: \"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0\") " pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:43.925353 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925086 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-sys\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.925353 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11dcfd6d-f84e-4c39-81a2-9149107e6b28-tmp\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.925353 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-sys-fs\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.925353 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925192 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:43.925353 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0874ec54-356c-4da7-9866-b10014ab9c77-iptables-alerter-script\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:43.925353 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13e9d2e6-45f5-4e22-99b9-c8ce050e23b0-agent-certs\") pod \"konnectivity-agent-zfb55\" (UID: \"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0\") " pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925366 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6fk9\" (UniqueName: \"kubernetes.io/projected/644b681a-6015-4423-889f-120d814f62e0-kube-api-access-l6fk9\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925411 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-etc-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925431 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-cni-bin\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/07efbfcf-2445-4754-93af-05c5041397b6-multus-daemon-config\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-etc-kubernetes\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:43.925601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925580 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-etc-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-host\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-slash\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-env-overrides\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-multus-certs\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925786 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-slash\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925827 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-multus-certs\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.925876 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925867 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-run\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-tuned\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-registration-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-cnibin\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.925989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-ovn\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovnkube-script-lib\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07efbfcf-2445-4754-93af-05c5041397b6-cni-binary-copy\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-socket-dir-parent\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6dcab854-eee8-4d5d-a448-602298a4e71d-hosts-file\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.926158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-var-lib-kubelet\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926264 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-env-overrides\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-cni-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-socket-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-dbus\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovnkube-config\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-cni-bin\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.926586 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-systemd\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926626 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-729kl\" (UniqueName: \"kubernetes.io/projected/87f2a6f9-bda9-494a-beb3-ea35876802fa-kube-api-access-729kl\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-netns\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-ovn\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbx8h\" (UniqueName: \"kubernetes.io/projected/07efbfcf-2445-4754-93af-05c5041397b6-kube-api-access-gbx8h\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-kubernetes\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926825 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6dcab854-eee8-4d5d-a448-602298a4e71d-hosts-file\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.927083 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovnkube-script-lib\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927178 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-var-lib-cni-bin\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927272 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-systemd\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.926711 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927340 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovnkube-config\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927358 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927389 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-host-run-netns\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927747 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927505 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07efbfcf-2445-4754-93af-05c5041397b6-cni-binary-copy\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927747 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927550 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-cni-dir\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927747 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927601 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-run-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927747 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927658 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nnhv\" (UniqueName: \"kubernetes.io/projected/11dcfd6d-f84e-4c39-81a2-9149107e6b28-kube-api-access-7nnhv\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:43.927747 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927709 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/07efbfcf-2445-4754-93af-05c5041397b6-multus-socket-dir-parent\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.927945 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87f2a6f9-bda9-494a-beb3-ea35876802fa-ovn-node-metrics-cert\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.927945 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz8bw\" (UniqueName: \"kubernetes.io/projected/5f873892-b168-498f-b71d-85f17e60c8ae-kube-api-access-nz8bw\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:43.927945 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-system-cni-dir\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:43.927945 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927915 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0874ec54-356c-4da7-9866-b10014ab9c77-host-slash\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:43.927945 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jtqp\" (UniqueName: \"kubernetes.io/projected/0874ec54-356c-4da7-9866-b10014ab9c77-kube-api-access-2jtqp\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:43.928158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.927971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-kubelet\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.928158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.928001 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-host-kubelet\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.928158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.928015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-var-lib-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.928158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.928072 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-var-lib-openvswitch\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.928158 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.928103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-node-log\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.928405 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.928162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87f2a6f9-bda9-494a-beb3-ea35876802fa-node-log\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.930435 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.930408 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:58:43.930435 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.930437 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:58:43.930610 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.930452 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:43.930610 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:43.930529 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:58:44.430507829 +0000 UTC m=+3.025266980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:43.932632 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.932606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcghm\" (UniqueName: \"kubernetes.io/projected/6dcab854-eee8-4d5d-a448-602298a4e71d-kube-api-access-mcghm\") pod \"node-resolver-vp854\" (UID: \"6dcab854-eee8-4d5d-a448-602298a4e71d\") " pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:43.932754 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.932653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mndzw\" (UniqueName: \"kubernetes.io/projected/d51c526d-3e94-4c65-821c-ae2b5466a4bb-kube-api-access-mndzw\") pod \"node-ca-m9qs8\" (UID: \"d51c526d-3e94-4c65-821c-ae2b5466a4bb\") " pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:43.935096 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.935065 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbx8h\" (UniqueName: \"kubernetes.io/projected/07efbfcf-2445-4754-93af-05c5041397b6-kube-api-access-gbx8h\") pod \"multus-qxzqd\" (UID: \"07efbfcf-2445-4754-93af-05c5041397b6\") " pod="openshift-multus/multus-qxzqd" Jul 07 21:58:43.935201 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.935158 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-729kl\" (UniqueName: \"kubernetes.io/projected/87f2a6f9-bda9-494a-beb3-ea35876802fa-kube-api-access-729kl\") pod \"ovnkube-node-69lbw\" (UID: \"87f2a6f9-bda9-494a-beb3-ea35876802fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:43.987441 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:43.987413 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 21:58:44.028877 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.028845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysctl-d\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.028877 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.028878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-modprobe-d\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.028901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-lib-modules\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.028985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-systemd\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-modprobe-d\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029025 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysctl-d\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029055 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-lib-modules\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029070 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-systemd\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-kubelet-config\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:44.029114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029113 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13e9d2e6-45f5-4e22-99b9-c8ce050e23b0-konnectivity-ca\") pod \"konnectivity-agent-zfb55\" (UID: \"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0\") " pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029075 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-sys\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029164 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-kubelet-config\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11dcfd6d-f84e-4c39-81a2-9149107e6b28-tmp\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029176 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-sys\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-sys-fs\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0874ec54-356c-4da7-9866-b10014ab9c77-iptables-alerter-script\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13e9d2e6-45f5-4e22-99b9-c8ce050e23b0-agent-certs\") pod \"konnectivity-agent-zfb55\" (UID: \"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0\") " pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6fk9\" (UniqueName: \"kubernetes.io/projected/644b681a-6015-4423-889f-120d814f62e0-kube-api-access-l6fk9\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029263 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-sys-fs\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.029369 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-host\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.029442 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:58:44.529425783 +0000 UTC m=+3.124184937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-run\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.029483 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:44.029570 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-tuned\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-registration-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-cnibin\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.029582 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:58:44.529568718 +0000 UTC m=+3.124327857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029605 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-cnibin\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-var-lib-kubelet\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029656 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029670 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-var-lib-kubelet\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-socket-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029705 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-host\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-dbus\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029745 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-registration-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-kubernetes\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0874ec54-356c-4da7-9866-b10014ab9c77-iptables-alerter-script\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.030537 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nnhv\" (UniqueName: \"kubernetes.io/projected/11dcfd6d-f84e-4c39-81a2-9149107e6b28-kube-api-access-7nnhv\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029799 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13e9d2e6-45f5-4e22-99b9-c8ce050e23b0-konnectivity-ca\") pod \"konnectivity-agent-zfb55\" (UID: \"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0\") " pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz8bw\" (UniqueName: \"kubernetes.io/projected/5f873892-b168-498f-b71d-85f17e60c8ae-kube-api-access-nz8bw\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-system-cni-dir\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.029897 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-dbus\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0874ec54-356c-4da7-9866-b10014ab9c77-host-slash\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jtqp\" (UniqueName: \"kubernetes.io/projected/0874ec54-356c-4da7-9866-b10014ab9c77-kube-api-access-2jtqp\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ng42\" (UniqueName: \"kubernetes.io/projected/a61b1873-a762-4e95-aa01-818198e15cc3-kube-api-access-8ng42\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030112 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-os-release\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-socket-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-device-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030161 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-kubernetes\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030188 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-device-dir\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysconfig\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysctl-conf\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030277 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030389 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5f873892-b168-498f-b71d-85f17e60c8ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030441 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysconfig\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030440 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-system-cni-dir\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030475 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-run\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030528 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0874ec54-356c-4da7-9866-b10014ab9c77-host-slash\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030552 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-sysctl-conf\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030615 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-os-release\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030723 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.030951 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/644b681a-6015-4423-889f-120d814f62e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.031143 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/644b681a-6015-4423-889f-120d814f62e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.031822 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11dcfd6d-f84e-4c39-81a2-9149107e6b28-tmp\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.031897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.031874 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13e9d2e6-45f5-4e22-99b9-c8ce050e23b0-agent-certs\") pod \"konnectivity-agent-zfb55\" (UID: \"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0\") " pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:44.032521 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.032504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11dcfd6d-f84e-4c39-81a2-9149107e6b28-etc-tuned\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.039999 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.039936 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz8bw\" (UniqueName: \"kubernetes.io/projected/5f873892-b168-498f-b71d-85f17e60c8ae-kube-api-access-nz8bw\") pod \"aws-ebs-csi-driver-node-cspf4\" (UID: \"5f873892-b168-498f-b71d-85f17e60c8ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.039999 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.039985 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nnhv\" (UniqueName: \"kubernetes.io/projected/11dcfd6d-f84e-4c39-81a2-9149107e6b28-kube-api-access-7nnhv\") pod \"tuned-7fgv5\" (UID: \"11dcfd6d-f84e-4c39-81a2-9149107e6b28\") " pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.040506 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.040486 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jtqp\" (UniqueName: \"kubernetes.io/projected/0874ec54-356c-4da7-9866-b10014ab9c77-kube-api-access-2jtqp\") pod \"iptables-alerter-265ks\" (UID: \"0874ec54-356c-4da7-9866-b10014ab9c77\") " pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.042092 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.042041 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6fk9\" (UniqueName: \"kubernetes.io/projected/644b681a-6015-4423-889f-120d814f62e0-kube-api-access-l6fk9\") pod \"multus-additional-cni-plugins-wc8mv\" (UID: \"644b681a-6015-4423-889f-120d814f62e0\") " pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.042168 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.042088 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ng42\" (UniqueName: \"kubernetes.io/projected/a61b1873-a762-4e95-aa01-818198e15cc3-kube-api-access-8ng42\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:44.116588 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.116559 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:58:44.126419 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.126392 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qxzqd" Jul 07 21:58:44.132938 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.132918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vp854" Jul 07 21:58:44.141496 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.141469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9qs8" Jul 07 21:58:44.149128 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.149106 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-265ks" Jul 07 21:58:44.155719 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.155694 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:58:44.161293 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.161272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" Jul 07 21:58:44.166859 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.166838 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" Jul 07 21:58:44.172299 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.172275 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" Jul 07 21:58:44.433023 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.432947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:44.433157 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.433090 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:58:44.433157 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.433109 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:58:44.433157 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.433121 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:44.433281 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.433176 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:58:45.433162816 +0000 UTC m=+4.027921948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:44.533934 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.533905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:44.534111 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.533959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:44.534111 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.534056 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:44.534111 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.534091 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:44.534230 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.534119 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:58:45.534100615 +0000 UTC m=+4.128859759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:44.534230 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.534151 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:58:45.534124923 +0000 UTC m=+4.128884074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:44.664988 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.664947 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e9d2e6_45f5_4e22_99b9_c8ce050e23b0.slice/crio-6d094c5966121f303840d09c207a0d6c07e604dc8313922287f4073dac427e46 WatchSource:0}: Error finding container 6d094c5966121f303840d09c207a0d6c07e604dc8313922287f4073dac427e46: Status 404 returned error can't find the container with id 6d094c5966121f303840d09c207a0d6c07e604dc8313922287f4073dac427e46 Jul 07 21:58:44.665803 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.665779 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd51c526d_3e94_4c65_821c_ae2b5466a4bb.slice/crio-5344ec07bd29020af819ed8a5db258cd566500c6c7a2e76a5885a1de5ca1b571 WatchSource:0}: Error finding container 5344ec07bd29020af819ed8a5db258cd566500c6c7a2e76a5885a1de5ca1b571: Status 404 returned error can't find the container with id 5344ec07bd29020af819ed8a5db258cd566500c6c7a2e76a5885a1de5ca1b571 Jul 07 21:58:44.667692 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.667247 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0874ec54_356c_4da7_9866_b10014ab9c77.slice/crio-a7075968d16385d357fbcd937473095259b0a43759915731aa5df833e65789af WatchSource:0}: Error finding container a7075968d16385d357fbcd937473095259b0a43759915731aa5df833e65789af: Status 404 returned error can't find the container with id a7075968d16385d357fbcd937473095259b0a43759915731aa5df833e65789af Jul 07 21:58:44.671063 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.671038 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f2a6f9_bda9_494a_beb3_ea35876802fa.slice/crio-5871c964b391b889f153a855bcb551f58a413c61ea21c4d368c8426a77906041 WatchSource:0}: Error finding container 5871c964b391b889f153a855bcb551f58a413c61ea21c4d368c8426a77906041: Status 404 returned error can't find the container with id 5871c964b391b889f153a855bcb551f58a413c61ea21c4d368c8426a77906041 Jul 07 21:58:44.672547 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.672487 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f873892_b168_498f_b71d_85f17e60c8ae.slice/crio-59f6c0adf51ffdf4a4d6baaf86a21fdf745d9379c43998b42f37f952fba0c854 WatchSource:0}: Error finding container 59f6c0adf51ffdf4a4d6baaf86a21fdf745d9379c43998b42f37f952fba0c854: Status 404 returned error can't find the container with id 59f6c0adf51ffdf4a4d6baaf86a21fdf745d9379c43998b42f37f952fba0c854 Jul 07 21:58:44.673417 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.673308 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dcfd6d_f84e_4c39_81a2_9149107e6b28.slice/crio-5537068f0a79c045b892da8bf3fc28046943193199225b40e55b5d3b06991f34 WatchSource:0}: Error finding container 5537068f0a79c045b892da8bf3fc28046943193199225b40e55b5d3b06991f34: Status 404 returned error can't find the container with id 5537068f0a79c045b892da8bf3fc28046943193199225b40e55b5d3b06991f34 Jul 07 21:58:44.674502 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.674477 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07efbfcf_2445_4754_93af_05c5041397b6.slice/crio-cf4db9c05deeeebc7900c8c3d781f8b35b5bf9183f5adf99fc575435285153e8 WatchSource:0}: Error finding container cf4db9c05deeeebc7900c8c3d781f8b35b5bf9183f5adf99fc575435285153e8: Status 404 returned error can't find the container with id cf4db9c05deeeebc7900c8c3d781f8b35b5bf9183f5adf99fc575435285153e8 Jul 07 21:58:44.675128 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.675048 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644b681a_6015_4423_889f_120d814f62e0.slice/crio-d6e029e57dff534aefab7bd2355bd7785958b6bf155106f38be2dfa8c7ce1d4d WatchSource:0}: Error finding container d6e029e57dff534aefab7bd2355bd7785958b6bf155106f38be2dfa8c7ce1d4d: Status 404 returned error can't find the container with id d6e029e57dff534aefab7bd2355bd7785958b6bf155106f38be2dfa8c7ce1d4d Jul 07 21:58:44.676603 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:58:44.676430 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcab854_eee8_4d5d_a448_602298a4e71d.slice/crio-5a059bec695fbadca7635fbedc37b0970d440e8fe66e9b929f48824643441e0c WatchSource:0}: Error finding container 5a059bec695fbadca7635fbedc37b0970d440e8fe66e9b929f48824643441e0c: Status 404 returned error can't find the container with id 5a059bec695fbadca7635fbedc37b0970d440e8fe66e9b929f48824643441e0c Jul 07 21:58:44.861662 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.861581 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 21:53:43 +0000 UTC" deadline="2028-01-03 11:15:49.653731625 +0000 UTC" Jul 07 21:58:44.861662 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.861661 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13069h17m4.79207803s" Jul 07 21:58:44.948993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.948966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:44.949095 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:44.949071 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:44.961187 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.961149 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" event={"ID":"74f5d17caa4e3653403abe6e004ae3b2","Type":"ContainerStarted","Data":"e4cc44540c16c146ac45343fc63644c5cd40d1520d7394b8ff24075af1da6c0f"} Jul 07 21:58:44.962148 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.962121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vp854" event={"ID":"6dcab854-eee8-4d5d-a448-602298a4e71d","Type":"ContainerStarted","Data":"5a059bec695fbadca7635fbedc37b0970d440e8fe66e9b929f48824643441e0c"} Jul 07 21:58:44.963031 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.963010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qxzqd" event={"ID":"07efbfcf-2445-4754-93af-05c5041397b6","Type":"ContainerStarted","Data":"cf4db9c05deeeebc7900c8c3d781f8b35b5bf9183f5adf99fc575435285153e8"} Jul 07 21:58:44.963981 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.963957 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" event={"ID":"11dcfd6d-f84e-4c39-81a2-9149107e6b28","Type":"ContainerStarted","Data":"5537068f0a79c045b892da8bf3fc28046943193199225b40e55b5d3b06991f34"} Jul 07 21:58:44.965324 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.965302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-265ks" event={"ID":"0874ec54-356c-4da7-9866-b10014ab9c77","Type":"ContainerStarted","Data":"a7075968d16385d357fbcd937473095259b0a43759915731aa5df833e65789af"} Jul 07 21:58:44.966450 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.966425 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9qs8" event={"ID":"d51c526d-3e94-4c65-821c-ae2b5466a4bb","Type":"ContainerStarted","Data":"5344ec07bd29020af819ed8a5db258cd566500c6c7a2e76a5885a1de5ca1b571"} Jul 07 21:58:44.967492 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.967469 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zfb55" event={"ID":"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0","Type":"ContainerStarted","Data":"6d094c5966121f303840d09c207a0d6c07e604dc8313922287f4073dac427e46"} Jul 07 21:58:44.968961 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.968938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" event={"ID":"3ebaa656986e8d73fd90f95493cd3249","Type":"ContainerStarted","Data":"d566eb6064646dbd48df85e0522958faea752696f6a67e0cd08e4f8615392b43"} Jul 07 21:58:44.970156 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.970136 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerStarted","Data":"d6e029e57dff534aefab7bd2355bd7785958b6bf155106f38be2dfa8c7ce1d4d"} Jul 07 21:58:44.971277 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.971240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" event={"ID":"5f873892-b168-498f-b71d-85f17e60c8ae","Type":"ContainerStarted","Data":"59f6c0adf51ffdf4a4d6baaf86a21fdf745d9379c43998b42f37f952fba0c854"} Jul 07 21:58:44.972436 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.972415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"5871c964b391b889f153a855bcb551f58a413c61ea21c4d368c8426a77906041"} Jul 07 21:58:44.998014 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:44.997954 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-125.ec2.internal" podStartSLOduration=1.997921016 podStartE2EDuration="1.997921016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:58:44.977469109 +0000 UTC m=+3.572228267" watchObservedRunningTime="2026-07-07 21:58:44.997921016 +0000 UTC m=+3.592680175" Jul 07 21:58:45.441820 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.441172 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:45.441820 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.441376 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:58:45.441820 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.441396 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:58:45.441820 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.441410 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:45.441820 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.441473 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:58:47.441453666 +0000 UTC m=+6.036212816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:45.542469 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.542432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:45.542602 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.542530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:45.542696 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.542674 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:45.542773 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.542734 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:58:47.542717048 +0000 UTC m=+6.137476204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:45.543472 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.543122 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:45.543472 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.543178 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:58:47.543161605 +0000 UTC m=+6.137920745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:45.862758 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.862665 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 21:53:43 +0000 UTC" deadline="2028-03-19 14:21:03.533600896 +0000 UTC" Jul 07 21:58:45.862758 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.862707 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14896h22m17.670898681s" Jul 07 21:58:45.951562 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.951523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:45.951732 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.951664 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:45.952116 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.952076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:45.952206 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:45.952183 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:45.993162 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.993120 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ebaa656986e8d73fd90f95493cd3249" containerID="d566eb6064646dbd48df85e0522958faea752696f6a67e0cd08e4f8615392b43" exitCode=0 Jul 07 21:58:45.994321 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.994294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" event={"ID":"3ebaa656986e8d73fd90f95493cd3249","Type":"ContainerDied","Data":"d566eb6064646dbd48df85e0522958faea752696f6a67e0cd08e4f8615392b43"} Jul 07 21:58:45.994466 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:45.994362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" event={"ID":"3ebaa656986e8d73fd90f95493cd3249","Type":"ContainerStarted","Data":"b7d6d7c1c3f78c785054baa70963f390ea4b5e1d03f49d3ee9bce51fb413499c"} Jul 07 21:58:46.949727 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:46.948997 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:46.949727 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:46.949145 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:47.464262 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:47.464218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:47.464483 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.464426 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:58:47.464483 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.464447 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:58:47.464483 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.464459 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:47.464633 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.464519 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:58:51.46449925 +0000 UTC m=+10.059258405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:47.565823 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:47.565211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:47.565823 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:47.565289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:47.565823 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.565429 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:47.565823 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.565429 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:47.565823 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.565491 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:58:51.565472757 +0000 UTC m=+10.160231907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:47.565823 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.565512 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:58:51.56550026 +0000 UTC m=+10.160259396 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:47.950718 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:47.950629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:47.951154 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.950752 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:47.951154 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:47.951146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:47.951266 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:47.951251 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:48.949534 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:48.949495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:48.949692 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:48.949635 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:49.949381 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:49.949346 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:49.949793 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:49.949495 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:49.949991 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:49.949969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:49.950122 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:49.950078 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:50.949778 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:50.949740 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:50.950200 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:50.949858 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:51.501748 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:51.501705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:51.501948 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.501916 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:58:51.501948 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.501940 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:58:51.502050 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.501953 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:51.502050 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.502006 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:58:59.501992339 +0000 UTC m=+18.096751487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:51.602713 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:51.602670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:51.602908 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:51.602769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:51.602908 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.602847 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:51.602908 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.602874 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:51.603072 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.602927 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:58:59.602909534 +0000 UTC m=+18.197668673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:51.603072 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.602943 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:58:59.602936259 +0000 UTC m=+18.197695394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:51.951544 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:51.951460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:51.952697 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.952315 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:51.952697 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:51.952424 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:51.952697 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:51.952514 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:52.948938 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:52.948896 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:52.949100 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:52.949045 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:53.949809 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:53.949424 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:53.949809 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:53.949537 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:53.949809 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:53.949563 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:53.949809 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:53.949648 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:54.948891 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:54.948845 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:54.949071 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:54.949006 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:55.949598 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:55.949516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:55.950004 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:55.949517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:55.950004 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:55.949664 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:55.950004 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:55.949745 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:56.949449 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:56.949408 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:56.949640 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:56.949530 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:57.951707 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:57.951677 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:57.952111 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:57.951677 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:57.952111 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:57.951817 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:58:57.952111 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:57.951871 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:58.948539 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:58.948510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:58.948820 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:58.948635 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:58:59.562447 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:59.562405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:58:59.562910 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.562577 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:58:59.562910 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.562594 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:58:59.562910 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.562609 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:59.562910 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.562689 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:15.562658322 +0000 UTC m=+34.157417479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:58:59.663524 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:59.663483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:59.663715 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:59.663539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:59.663715 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.663602 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:59.663715 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.663655 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:59.663715 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.663679 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:59:15.663664024 +0000 UTC m=+34.258423158 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:58:59.663715 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.663699 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:59:15.6636869 +0000 UTC m=+34.258446034 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:58:59.949364 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:59.948852 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:58:59.949364 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.948985 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:58:59.949364 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:58:59.949054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:58:59.949364 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:58:59.949182 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:00.948690 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:00.948655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:00.949083 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:00.948788 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:01.949550 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:01.949509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:01.949998 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:01.949598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:01.949998 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:01.949627 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:01.949998 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:01.949710 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:02.949136 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:02.949096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:02.949343 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:02.949236 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:03.950973 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:03.950487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:03.951675 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:03.950629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:03.951675 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:03.951078 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:03.951675 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:03.951166 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:04.037211 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.037132 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vp854" event={"ID":"6dcab854-eee8-4d5d-a448-602298a4e71d","Type":"ContainerStarted","Data":"11c52f1a74f649d13684b3be94f709fa1ebd7183c9a45d16c8dd452480505684"} Jul 07 21:59:04.038807 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.038777 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qxzqd" event={"ID":"07efbfcf-2445-4754-93af-05c5041397b6","Type":"ContainerStarted","Data":"52b71fdecb7e8ceaa2ba80ad51027dad06d903dc1ebb58c53513da4cbaced267"} Jul 07 21:59:04.040713 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.040684 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" event={"ID":"11dcfd6d-f84e-4c39-81a2-9149107e6b28","Type":"ContainerStarted","Data":"fbdc5c9d25ab5ba9a220f50f6793caaa29e5328db51670d64094a2d55698d946"} Jul 07 21:59:04.042507 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.042477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9qs8" event={"ID":"d51c526d-3e94-4c65-821c-ae2b5466a4bb","Type":"ContainerStarted","Data":"64e0c38f7320d329b7437a39f1be82caaf691598624daf4aabdd44e75bf6e990"} Jul 07 21:59:04.044094 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.044072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zfb55" event={"ID":"13e9d2e6-45f5-4e22-99b9-c8ce050e23b0","Type":"ContainerStarted","Data":"75c21e28fe1d345a1d90845e3463ca672e7ebc3fc4b1fbb090192f9ecd796d4f"} Jul 07 21:59:04.045726 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.045700 2562 generic.go:356] "Generic (PLEG): container finished" podID="644b681a-6015-4423-889f-120d814f62e0" containerID="8be304f7c061e19a070f0fa6d776cf38029e72a109de283abf752441e100be78" exitCode=0 Jul 07 21:59:04.045833 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.045780 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerDied","Data":"8be304f7c061e19a070f0fa6d776cf38029e72a109de283abf752441e100be78"} Jul 07 21:59:04.047553 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.047364 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" event={"ID":"5f873892-b168-498f-b71d-85f17e60c8ae","Type":"ContainerStarted","Data":"98862a0ac82fa431dd72919a41c8c85c2dd928a7128aba998ed679f589e27db4"} Jul 07 21:59:04.049861 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.049839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"9202e6366648340ba6794be8540a921aac7381af6e9f9d7a752ace80096533e1"} Jul 07 21:59:04.049944 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.049904 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"51ce2b9044da62377f87fc6773671f8e542a00f8269e4e17572d63eefa5ddb3b"} Jul 07 21:59:04.057133 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.057087 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-125.ec2.internal" podStartSLOduration=21.057067765 podStartE2EDuration="21.057067765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:58:46.013968065 +0000 UTC m=+4.608727223" watchObservedRunningTime="2026-07-07 21:59:04.057067765 +0000 UTC m=+22.651826923" Jul 07 21:59:04.077214 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.077164 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7fgv5" podStartSLOduration=4.0673152 podStartE2EDuration="23.077145417s" totalImagesPullingTime="19.009830217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:04.076827278 +0000 UTC m=+22.671586435" watchObservedRunningTime="2026-07-07 21:59:04.077145417 +0000 UTC m=+22.671904643" Jul 07 21:59:04.077398 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.077261 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vp854" podStartSLOduration=4.104775199 podStartE2EDuration="23.077256827s" totalImagesPullingTime="18.972481628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:04.05720995 +0000 UTC m=+22.651969087" watchObservedRunningTime="2026-07-07 21:59:04.077256827 +0000 UTC m=+22.672015984" Jul 07 21:59:04.156001 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.155923 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m9qs8" podStartSLOduration=4.17593064 podStartE2EDuration="23.155906689s" totalImagesPullingTime="18.979976049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:04.155899557 +0000 UTC m=+22.750658714" watchObservedRunningTime="2026-07-07 21:59:04.155906689 +0000 UTC m=+22.750665850" Jul 07 21:59:04.156478 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.156451 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qxzqd" podStartSLOduration=4.122088774 podStartE2EDuration="23.156439871s" totalImagesPullingTime="19.034351097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:04.127439033 +0000 UTC m=+22.722198190" watchObservedRunningTime="2026-07-07 21:59:04.156439871 +0000 UTC m=+22.751199028" Jul 07 21:59:04.177516 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.177474 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zfb55" podStartSLOduration=4.221318701 podStartE2EDuration="23.177458114s" totalImagesPullingTime="18.956139413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:04.177011928 +0000 UTC m=+22.771771079" watchObservedRunningTime="2026-07-07 21:59:04.177458114 +0000 UTC m=+22.772217270" Jul 07 21:59:04.949607 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:04.949525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:04.949793 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:04.949659 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:05.055646 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.055586 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"ebeb9f9dad19ad16d983939b9e4de33a5060e5f541ffaf8d16ae24d4164ed52d"} Jul 07 21:59:05.056194 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.055684 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"839eb7bd3c321666aa730f27f321f4065ed592536901f271700f9032fe27f57d"} Jul 07 21:59:05.056194 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.055717 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"7c82cdf70df52a7ed642e2fcc96744fdbd559900febc14555674deb598c687de"} Jul 07 21:59:05.056194 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.055764 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"44c0462bd9052da3a2640b1fd2257711360c5027bcc4c4486e3eedda70e1fc04"} Jul 07 21:59:05.385359 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.385152 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 21:59:05.882982 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.882884 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T21:59:05.385171677Z","UUID":"58f9921b-8dd8-4432-a182-b988833943d6","Handler":null,"Name":"","Endpoint":""} Jul 07 21:59:05.884704 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.884680 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 21:59:05.884839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.884718 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 21:59:05.949517 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.948559 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:05.949517 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:05.948759 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:05.949517 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:05.949362 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:05.949517 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:05.949446 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:06.059354 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:06.059286 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-265ks" event={"ID":"0874ec54-356c-4da7-9866-b10014ab9c77","Type":"ContainerStarted","Data":"8435a8018e7af79dfcccbc86ba833786c229685b5effa9b5814cf3ee7c67792f"} Jul 07 21:59:06.061211 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:06.061179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" event={"ID":"5f873892-b168-498f-b71d-85f17e60c8ae","Type":"ContainerStarted","Data":"ff4e42f7b5746ca35833d46ba3e49102e7cfaac02ae33fa6861321ec614c572e"} Jul 07 21:59:06.949038 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:06.948784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:06.949259 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:06.949090 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:07.066441 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:07.066396 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" event={"ID":"5f873892-b168-498f-b71d-85f17e60c8ae","Type":"ContainerStarted","Data":"6a425a0d9ad6962fc143574d4b96d358875e3c6a0bced68d56e803ad206f4e2c"} Jul 07 21:59:07.070121 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:07.070087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"0251b5856dbffd8babbb5050bbd7abf725088b2310bcb3319c8c5a541d99ea06"} Jul 07 21:59:07.087344 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:07.087238 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-265ks" podStartSLOduration=7.109094221 podStartE2EDuration="26.087215726s" totalImagesPullingTime="18.978121505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:06.075888022 +0000 UTC m=+24.670647181" watchObservedRunningTime="2026-07-07 21:59:07.087215726 +0000 UTC m=+25.681974885" Jul 07 21:59:07.087652 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:07.087609 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cspf4" podStartSLOduration=4.593210071 podStartE2EDuration="26.08759684s" totalImagesPullingTime="21.494386769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:07.087077195 +0000 UTC m=+25.681836352" watchObservedRunningTime="2026-07-07 21:59:07.08759684 +0000 UTC m=+25.682355997" Jul 07 21:59:07.948559 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:07.948520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:07.948868 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:07.948574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:07.948868 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:07.948681 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:07.948868 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:07.948813 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:08.949689 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:08.949363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:08.950313 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:08.949739 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:09.040590 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.040547 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:59:09.041175 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.041153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:59:09.076298 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.076266 2562 generic.go:356] "Generic (PLEG): container finished" podID="644b681a-6015-4423-889f-120d814f62e0" containerID="e6b3e46b52bbf95803f334c2b311cd7e591f2222fba53cd446867843a8a10671" exitCode=0 Jul 07 21:59:09.076511 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.076362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerDied","Data":"e6b3e46b52bbf95803f334c2b311cd7e591f2222fba53cd446867843a8a10671"} Jul 07 21:59:09.080125 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.080092 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" event={"ID":"87f2a6f9-bda9-494a-beb3-ea35876802fa","Type":"ContainerStarted","Data":"5bedea33ca7488735ff3de9656ffa6fc508af6856915862f3522c6caab606835"} Jul 07 21:59:09.080221 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.080199 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:59:09.080724 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.080701 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zfb55" Jul 07 21:59:09.139256 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.139208 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" podStartSLOduration=9.087751382 podStartE2EDuration="28.139194029s" totalImagesPullingTime="19.051442647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:09.138893666 +0000 UTC m=+27.733652823" watchObservedRunningTime="2026-07-07 21:59:09.139194029 +0000 UTC m=+27.733953185" Jul 07 21:59:09.948646 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.948605 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:09.948913 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:09.948738 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:09.948913 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:09.948788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:09.948913 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:09.948875 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:10.082798 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:10.082766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:59:10.083288 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:10.082856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:59:10.083288 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:10.082904 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:59:10.105378 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:10.105244 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:59:10.109980 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:10.109952 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:59:10.949040 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:10.949006 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:10.949237 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:10.949111 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:11.008907 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.008876 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l6chz"] Jul 07 21:59:11.009098 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.009001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:11.009176 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:11.009116 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:11.010918 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.010895 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-htffx"] Jul 07 21:59:11.011023 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.010991 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:11.011098 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:11.011081 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:11.024765 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.024733 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cffq9"] Jul 07 21:59:11.086532 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.086438 2562 generic.go:356] "Generic (PLEG): container finished" podID="644b681a-6015-4423-889f-120d814f62e0" containerID="758fa88ceb52bc2ec987db190bd455a0b4dcfca2b02a94a2b6e1159fa05af366" exitCode=0 Jul 07 21:59:11.086968 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.086544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerDied","Data":"758fa88ceb52bc2ec987db190bd455a0b4dcfca2b02a94a2b6e1159fa05af366"} Jul 07 21:59:11.086968 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:11.086574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:11.086968 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:11.086697 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:12.948750 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:12.948549 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:12.949232 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:12.948571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:12.949232 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:12.948835 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:12.949232 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:12.948549 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:12.949232 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:12.948913 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:12.949232 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:12.949003 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:13.093627 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:13.093541 2562 generic.go:356] "Generic (PLEG): container finished" podID="644b681a-6015-4423-889f-120d814f62e0" containerID="66c1feaabe53f36fcd3bdd5ab4696e90a56248bfe7ae7d01540b5c01da0233f8" exitCode=0 Jul 07 21:59:13.093762 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:13.093621 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerDied","Data":"66c1feaabe53f36fcd3bdd5ab4696e90a56248bfe7ae7d01540b5c01da0233f8"} Jul 07 21:59:14.949985 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:14.949471 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:14.949985 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:14.949510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:14.949985 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:14.949601 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cffq9" podUID="5e8e2ad6-8c16-49de-a957-8bd1692fbf9c" Jul 07 21:59:14.949985 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:14.949833 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htffx" podUID="a61b1873-a762-4e95-aa01-818198e15cc3" Jul 07 21:59:14.949985 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:14.949480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:14.949985 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:14.949932 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6chz" podUID="9eba0b17-6bdb-4a42-ad6e-ba0d705772ef" Jul 07 21:59:15.588114 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:15.588068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:15.588280 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.588240 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 21:59:15.588280 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.588264 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 21:59:15.588280 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.588274 2562 projected.go:196] Error preparing data for projected volume kube-api-access-bfbn9 for pod openshift-network-diagnostics/network-check-target-cffq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:59:15.588420 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.588343 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9 podName:5e8e2ad6-8c16-49de-a957-8bd1692fbf9c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:47.588315934 +0000 UTC m=+66.183075088 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfbn9" (UniqueName: "kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9") pod "network-check-target-cffq9" (UID: "5e8e2ad6-8c16-49de-a957-8bd1692fbf9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 21:59:15.689097 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:15.689047 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:15.689287 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:15.689129 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:15.689287 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.689233 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:59:15.689287 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.689252 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 21:59:15.689454 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.689310 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs podName:a61b1873-a762-4e95-aa01-818198e15cc3 nodeName:}" failed. No retries permitted until 2026-07-07 21:59:47.689292726 +0000 UTC m=+66.284051866 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs") pod "network-metrics-daemon-htffx" (UID: "a61b1873-a762-4e95-aa01-818198e15cc3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 21:59:15.689454 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:15.689348 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret podName:9eba0b17-6bdb-4a42-ad6e-ba0d705772ef nodeName:}" failed. No retries permitted until 2026-07-07 21:59:47.689323137 +0000 UTC m=+66.284082286 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret") pod "global-pull-secret-syncer-l6chz" (UID: "9eba0b17-6bdb-4a42-ad6e-ba0d705772ef") : object "kube-system"/"original-pull-secret" not registered Jul 07 21:59:16.524631 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.524549 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-125.ec2.internal" event="NodeReady" Jul 07 21:59:16.525042 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.524714 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 21:59:16.564040 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.564005 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9c79b7c7b-54rf4"] Jul 07 21:59:16.599154 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.599113 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l4ptc"] Jul 07 21:59:16.599344 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.599246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.601963 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.601940 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 21:59:16.602119 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.601941 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzmhm\"" Jul 07 21:59:16.602119 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.601941 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 21:59:16.602119 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.601996 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 21:59:16.608341 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.608302 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 21:59:16.613197 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.613161 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tmvwz"] Jul 07 21:59:16.613341 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.613294 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.615787 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.615764 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 21:59:16.615897 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.615798 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6c5lw\"" Jul 07 21:59:16.616356 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.616318 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 21:59:16.629259 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.629235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9c79b7c7b-54rf4"] Jul 07 21:59:16.629259 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.629266 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l4ptc"] Jul 07 21:59:16.629466 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.629291 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tmvwz"] Jul 07 21:59:16.629466 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.629391 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.632038 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.632015 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-czs54\"" Jul 07 21:59:16.632038 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.632034 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 21:59:16.632232 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.632036 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 21:59:16.632529 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.632511 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 21:59:16.696301 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696259 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-installation-pull-secrets\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696301 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-bound-sa-token\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696603 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-trusted-ca\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696603 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smvpl\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-kube-api-access-smvpl\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696603 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-image-registry-private-configuration\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696603 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696777 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-certificates\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.696777 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.696677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f348a962-e8d6-4321-b95c-f0495c5c75bf-ca-trust-extracted\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.797571 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797474 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f348a962-e8d6-4321-b95c-f0495c5c75bf-ca-trust-extracted\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.797571 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/24de3685-1707-4673-9b91-33499812966c-config-volume\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.797571 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nzct\" (UniqueName: \"kubernetes.io/projected/24de3685-1707-4673-9b91-33499812966c-kube-api-access-6nzct\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24de3685-1707-4673-9b91-33499812966c-tmp-dir\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-installation-pull-secrets\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-bound-sa-token\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-trusted-ca\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smvpl\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-kube-api-access-smvpl\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.797839 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797831 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8x2p\" (UniqueName: \"kubernetes.io/projected/259d2c92-3c9b-4475-8693-e0eb6bd3969c-kube-api-access-d8x2p\") pod \"ingress-canary-tmvwz\" (UID: \"259d2c92-3c9b-4475-8693-e0eb6bd3969c\") " pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.798229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-image-registry-private-configuration\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.798229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.798229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-certificates\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.798229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/259d2c92-3c9b-4475-8693-e0eb6bd3969c-cert\") pod \"ingress-canary-tmvwz\" (UID: \"259d2c92-3c9b-4475-8693-e0eb6bd3969c\") " pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.798229 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.797905 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f348a962-e8d6-4321-b95c-f0495c5c75bf-ca-trust-extracted\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.798517 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:16.798492 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 21:59:16.798517 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:16.798511 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c79b7c7b-54rf4: secret "image-registry-tls" not found Jul 07 21:59:16.798615 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:16.798606 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls podName:f348a962-e8d6-4321-b95c-f0495c5c75bf nodeName:}" failed. No retries permitted until 2026-07-07 21:59:17.298587763 +0000 UTC m=+35.893346901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls") pod "image-registry-9c79b7c7b-54rf4" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf") : secret "image-registry-tls" not found Jul 07 21:59:16.798861 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.798835 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-certificates\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.798984 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.798955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-trusted-ca\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.802786 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.802648 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-installation-pull-secrets\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.802898 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.802716 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-image-registry-private-configuration\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.807446 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.807421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-bound-sa-token\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.807779 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.807758 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smvpl\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-kube-api-access-smvpl\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:16.898612 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.898572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/259d2c92-3c9b-4475-8693-e0eb6bd3969c-cert\") pod \"ingress-canary-tmvwz\" (UID: \"259d2c92-3c9b-4475-8693-e0eb6bd3969c\") " pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.898815 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.898651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/24de3685-1707-4673-9b91-33499812966c-config-volume\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.898815 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.898691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nzct\" (UniqueName: \"kubernetes.io/projected/24de3685-1707-4673-9b91-33499812966c-kube-api-access-6nzct\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.898925 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.898834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24de3685-1707-4673-9b91-33499812966c-tmp-dir\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.898925 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.898884 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.899008 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.898937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8x2p\" (UniqueName: \"kubernetes.io/projected/259d2c92-3c9b-4475-8693-e0eb6bd3969c-kube-api-access-d8x2p\") pod \"ingress-canary-tmvwz\" (UID: \"259d2c92-3c9b-4475-8693-e0eb6bd3969c\") " pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.899202 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:16.899184 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 21:59:16.899281 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:16.899268 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls podName:24de3685-1707-4673-9b91-33499812966c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:17.399249589 +0000 UTC m=+35.994008728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls") pod "dns-default-l4ptc" (UID: "24de3685-1707-4673-9b91-33499812966c") : secret "dns-default-metrics-tls" not found Jul 07 21:59:16.899367 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.899277 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24de3685-1707-4673-9b91-33499812966c-tmp-dir\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.899367 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.899286 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/24de3685-1707-4673-9b91-33499812966c-config-volume\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.903871 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.903843 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/259d2c92-3c9b-4475-8693-e0eb6bd3969c-cert\") pod \"ingress-canary-tmvwz\" (UID: \"259d2c92-3c9b-4475-8693-e0eb6bd3969c\") " pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.909814 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.909083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nzct\" (UniqueName: \"kubernetes.io/projected/24de3685-1707-4673-9b91-33499812966c-kube-api-access-6nzct\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:16.909814 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.909114 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8x2p\" (UniqueName: \"kubernetes.io/projected/259d2c92-3c9b-4475-8693-e0eb6bd3969c-kube-api-access-d8x2p\") pod \"ingress-canary-tmvwz\" (UID: \"259d2c92-3c9b-4475-8693-e0eb6bd3969c\") " pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.939295 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.939255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tmvwz" Jul 07 21:59:16.949582 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.949549 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:16.949766 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.949599 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:16.949766 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.949684 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:16.952783 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.952756 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 21:59:16.953015 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.952996 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b8gff\"" Jul 07 21:59:16.953196 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.953182 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 21:59:16.953464 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.953451 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqsmx\"" Jul 07 21:59:16.953628 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.953614 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 21:59:16.953819 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:16.953806 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 21:59:17.303286 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:17.303256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:17.303522 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:17.303468 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 21:59:17.303522 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:17.303493 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c79b7c7b-54rf4: secret "image-registry-tls" not found Jul 07 21:59:17.303624 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:17.303586 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls podName:f348a962-e8d6-4321-b95c-f0495c5c75bf nodeName:}" failed. No retries permitted until 2026-07-07 21:59:18.303564248 +0000 UTC m=+36.898323406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls") pod "image-registry-9c79b7c7b-54rf4" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf") : secret "image-registry-tls" not found Jul 07 21:59:17.372668 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:17.372635 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tmvwz"] Jul 07 21:59:17.404061 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:17.404029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:17.404219 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:17.404199 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 21:59:17.404283 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:17.404273 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls podName:24de3685-1707-4673-9b91-33499812966c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:18.40425763 +0000 UTC m=+36.999016765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls") pod "dns-default-l4ptc" (UID: "24de3685-1707-4673-9b91-33499812966c") : secret "dns-default-metrics-tls" not found Jul 07 21:59:17.457967 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:17.457927 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259d2c92_3c9b_4475_8693_e0eb6bd3969c.slice/crio-ea4e0663413b709c61f3b1d47e37357bea3cdeb41202191e0675ace776a4994d WatchSource:0}: Error finding container ea4e0663413b709c61f3b1d47e37357bea3cdeb41202191e0675ace776a4994d: Status 404 returned error can't find the container with id ea4e0663413b709c61f3b1d47e37357bea3cdeb41202191e0675ace776a4994d Jul 07 21:59:18.106715 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:18.106672 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tmvwz" event={"ID":"259d2c92-3c9b-4475-8693-e0eb6bd3969c","Type":"ContainerStarted","Data":"ea4e0663413b709c61f3b1d47e37357bea3cdeb41202191e0675ace776a4994d"} Jul 07 21:59:18.109791 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:18.109757 2562 generic.go:356] "Generic (PLEG): container finished" podID="644b681a-6015-4423-889f-120d814f62e0" containerID="20aa1ea4c25fd8c23dffaee6c104260a30ee8cf57048f0db807568ed6a2fd058" exitCode=0 Jul 07 21:59:18.109941 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:18.109822 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerDied","Data":"20aa1ea4c25fd8c23dffaee6c104260a30ee8cf57048f0db807568ed6a2fd058"} Jul 07 21:59:18.311076 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:18.311040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:18.311239 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:18.311185 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 21:59:18.311239 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:18.311200 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c79b7c7b-54rf4: secret "image-registry-tls" not found Jul 07 21:59:18.311371 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:18.311261 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls podName:f348a962-e8d6-4321-b95c-f0495c5c75bf nodeName:}" failed. No retries permitted until 2026-07-07 21:59:20.311241382 +0000 UTC m=+38.906000540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls") pod "image-registry-9c79b7c7b-54rf4" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf") : secret "image-registry-tls" not found Jul 07 21:59:18.411970 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:18.411881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:18.412121 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:18.412028 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 21:59:18.412121 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:18.412093 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls podName:24de3685-1707-4673-9b91-33499812966c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:20.412073889 +0000 UTC m=+39.006833030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls") pod "dns-default-l4ptc" (UID: "24de3685-1707-4673-9b91-33499812966c") : secret "dns-default-metrics-tls" not found Jul 07 21:59:19.116201 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:19.115946 2562 generic.go:356] "Generic (PLEG): container finished" podID="644b681a-6015-4423-889f-120d814f62e0" containerID="1d54d9d48997b654148125e9fb7b1fbc772a591071552fdee9519cf57e46e57a" exitCode=0 Jul 07 21:59:19.116638 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:19.116026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerDied","Data":"1d54d9d48997b654148125e9fb7b1fbc772a591071552fdee9519cf57e46e57a"} Jul 07 21:59:20.120270 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:20.120223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tmvwz" event={"ID":"259d2c92-3c9b-4475-8693-e0eb6bd3969c","Type":"ContainerStarted","Data":"abf9fa1979e13fb8aaf616d0380de1132379141f45b515043d020e76d648ec09"} Jul 07 21:59:20.123487 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:20.123456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" event={"ID":"644b681a-6015-4423-889f-120d814f62e0","Type":"ContainerStarted","Data":"725cd228ca0427324c5f39ddd231205e34233e6240d73073909ad873c768883f"} Jul 07 21:59:20.141167 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:20.141104 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tmvwz" podStartSLOduration=1.884016099 podStartE2EDuration="4.141081748s" totalImagesPullingTime="2.257065649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:20.140531925 +0000 UTC m=+38.735291082" watchObservedRunningTime="2026-07-07 21:59:20.141081748 +0000 UTC m=+38.735840918" Jul 07 21:59:20.186361 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:20.186295 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wc8mv" podStartSLOduration=8.926678738 podStartE2EDuration="39.186276509s" totalImagesPullingTime="30.259597771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:20.183782476 +0000 UTC m=+38.778541632" watchObservedRunningTime="2026-07-07 21:59:20.186276509 +0000 UTC m=+38.781035666" Jul 07 21:59:20.327160 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:20.327113 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:20.327367 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:20.327262 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 21:59:20.327367 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:20.327279 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c79b7c7b-54rf4: secret "image-registry-tls" not found Jul 07 21:59:20.327442 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:20.327369 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls podName:f348a962-e8d6-4321-b95c-f0495c5c75bf nodeName:}" failed. No retries permitted until 2026-07-07 21:59:24.327320207 +0000 UTC m=+42.922079345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls") pod "image-registry-9c79b7c7b-54rf4" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf") : secret "image-registry-tls" not found Jul 07 21:59:20.428563 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:20.428533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:20.428693 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:20.428678 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 21:59:20.428748 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:20.428738 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls podName:24de3685-1707-4673-9b91-33499812966c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:24.428722974 +0000 UTC m=+43.023482110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls") pod "dns-default-l4ptc" (UID: "24de3685-1707-4673-9b91-33499812966c") : secret "dns-default-metrics-tls" not found Jul 07 21:59:22.086873 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.086840 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc"] Jul 07 21:59:22.088743 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.088728 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" Jul 07 21:59:22.092902 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.092881 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 21:59:22.093543 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.093525 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 21:59:22.093543 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.093534 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-c5f4g\"" Jul 07 21:59:22.117077 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.117040 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc"] Jul 07 21:59:22.144983 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.144950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtcgm\" (UniqueName: \"kubernetes.io/projected/d3a3a117-7d53-47c5-b30e-f9ab475065eb-kube-api-access-vtcgm\") pod \"volume-data-source-validator-5d456fdbcd-j7fhc\" (UID: \"d3a3a117-7d53-47c5-b30e-f9ab475065eb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" Jul 07 21:59:22.172504 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.172468 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p"] Jul 07 21:59:22.174195 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.174179 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.176399 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.176377 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z"] Jul 07 21:59:22.177645 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.177624 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 21:59:22.177779 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.177642 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 21:59:22.178111 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.178097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.178164 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.178130 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 21:59:22.178835 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.178817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9tqcz\"" Jul 07 21:59:22.183670 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.183650 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 21:59:22.183794 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.183684 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nnsmw\"" Jul 07 21:59:22.183945 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.183930 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 21:59:22.184167 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.184154 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 21:59:22.185520 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.185505 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 21:59:22.198106 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.198077 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p"] Jul 07 21:59:22.198992 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.198966 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z"] Jul 07 21:59:22.245514 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.245481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.245721 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.245532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnq5g\" (UniqueName: \"kubernetes.io/projected/494370d8-286d-4c39-9b7c-15995a13779d-kube-api-access-pnq5g\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.245721 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.245577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtcgm\" (UniqueName: \"kubernetes.io/projected/d3a3a117-7d53-47c5-b30e-f9ab475065eb-kube-api-access-vtcgm\") pod \"volume-data-source-validator-5d456fdbcd-j7fhc\" (UID: \"d3a3a117-7d53-47c5-b30e-f9ab475065eb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" Jul 07 21:59:22.245721 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.245615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/494370d8-286d-4c39-9b7c-15995a13779d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.258706 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.258669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtcgm\" (UniqueName: \"kubernetes.io/projected/d3a3a117-7d53-47c5-b30e-f9ab475065eb-kube-api-access-vtcgm\") pod \"volume-data-source-validator-5d456fdbcd-j7fhc\" (UID: \"d3a3a117-7d53-47c5-b30e-f9ab475065eb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" Jul 07 21:59:22.279318 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.279280 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-vbppw"] Jul 07 21:59:22.281159 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.281136 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w"] Jul 07 21:59:22.281324 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.281278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.282892 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.282874 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" Jul 07 21:59:22.284054 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.284033 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 21:59:22.284146 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.284035 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 21:59:22.285095 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.285079 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-68bk7\"" Jul 07 21:59:22.285856 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.285840 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4wwtq\"" Jul 07 21:59:22.314145 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.314072 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-vbppw"] Jul 07 21:59:22.315072 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.315052 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w"] Jul 07 21:59:22.346533 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.346533 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhm2\" (UniqueName: \"kubernetes.io/projected/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-kube-api-access-ghhm2\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.346533 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.346814 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.346814 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.346624 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:22.346814 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnq5g\" (UniqueName: \"kubernetes.io/projected/494370d8-286d-4c39-9b7c-15995a13779d-kube-api-access-pnq5g\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.346814 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.346684 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls podName:494370d8-286d-4c39-9b7c-15995a13779d nodeName:}" failed. No retries permitted until 2026-07-07 21:59:22.846669175 +0000 UTC m=+41.441428310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ct22z" (UID: "494370d8-286d-4c39-9b7c-15995a13779d") : secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:22.346814 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346714 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.346814 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.346755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/494370d8-286d-4c39-9b7c-15995a13779d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.347403 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.347386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/494370d8-286d-4c39-9b7c-15995a13779d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.363382 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.363321 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnq5g\" (UniqueName: \"kubernetes.io/projected/494370d8-286d-4c39-9b7c-15995a13779d-kube-api-access-pnq5g\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.398450 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.398403 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" Jul 07 21:59:22.447740 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.447711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.447899 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.447776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.447899 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.447881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs8sj\" (UniqueName: \"kubernetes.io/projected/e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa-kube-api-access-fs8sj\") pod \"network-check-source-989f8b8dc-rrm5w\" (UID: \"e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" Jul 07 21:59:22.447899 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.447886 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 21:59:22.448049 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.447939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhm2\" (UniqueName: \"kubernetes.io/projected/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-kube-api-access-ghhm2\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.448049 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.447973 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert podName:f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:22.94795116 +0000 UTC m=+41.542710315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vbppw" (UID: "f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c") : secret "networking-console-plugin-cert" not found Jul 07 21:59:22.448049 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.448000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.448345 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.448299 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 21:59:22.448434 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.448380 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls podName:f940dfb9-d9e0-4847-8eef-b03b13bf73ca nodeName:}" failed. No retries permitted until 2026-07-07 21:59:22.948364829 +0000 UTC m=+41.543123983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8q9p" (UID: "f940dfb9-d9e0-4847-8eef-b03b13bf73ca") : secret "samples-operator-tls" not found Jul 07 21:59:22.449047 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.449021 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.459818 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.459777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhm2\" (UniqueName: \"kubernetes.io/projected/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-kube-api-access-ghhm2\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.535887 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.535853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc"] Jul 07 21:59:22.538943 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:22.538899 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a3a117_7d53_47c5_b30e_f9ab475065eb.slice/crio-fe7c44e1b63c4ad0a3a1bb80f4deb00e375662748b41c880d1f41516f7f4d316 WatchSource:0}: Error finding container fe7c44e1b63c4ad0a3a1bb80f4deb00e375662748b41c880d1f41516f7f4d316: Status 404 returned error can't find the container with id fe7c44e1b63c4ad0a3a1bb80f4deb00e375662748b41c880d1f41516f7f4d316 Jul 07 21:59:22.548709 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.548683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs8sj\" (UniqueName: \"kubernetes.io/projected/e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa-kube-api-access-fs8sj\") pod \"network-check-source-989f8b8dc-rrm5w\" (UID: \"e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" Jul 07 21:59:22.561306 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.561276 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs8sj\" (UniqueName: \"kubernetes.io/projected/e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa-kube-api-access-fs8sj\") pod \"network-check-source-989f8b8dc-rrm5w\" (UID: \"e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" Jul 07 21:59:22.596197 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.596168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" Jul 07 21:59:22.729032 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.728935 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w"] Jul 07 21:59:22.731712 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:22.731681 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7065e6f_ac04_4ee5_bb1a_b1dc3b4ba4aa.slice/crio-f41322161c18fee51367bde31e3886c2b0df1ae60703c57a541a50a892cecb07 WatchSource:0}: Error finding container f41322161c18fee51367bde31e3886c2b0df1ae60703c57a541a50a892cecb07: Status 404 returned error can't find the container with id f41322161c18fee51367bde31e3886c2b0df1ae60703c57a541a50a892cecb07 Jul 07 21:59:22.850694 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.850609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:22.850846 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.850753 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:22.850846 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.850817 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls podName:494370d8-286d-4c39-9b7c-15995a13779d nodeName:}" failed. No retries permitted until 2026-07-07 21:59:23.850801138 +0000 UTC m=+42.445560274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ct22z" (UID: "494370d8-286d-4c39-9b7c-15995a13779d") : secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:22.951302 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.951263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:22.951497 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:22.951317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:22.951497 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.951431 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 21:59:22.951497 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.951479 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls podName:f940dfb9-d9e0-4847-8eef-b03b13bf73ca nodeName:}" failed. No retries permitted until 2026-07-07 21:59:23.951464681 +0000 UTC m=+42.546223817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8q9p" (UID: "f940dfb9-d9e0-4847-8eef-b03b13bf73ca") : secret "samples-operator-tls" not found Jul 07 21:59:22.951607 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.951539 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 21:59:22.951636 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:22.951610 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert podName:f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:23.951592511 +0000 UTC m=+42.546351647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vbppw" (UID: "f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c") : secret "networking-console-plugin-cert" not found Jul 07 21:59:23.131743 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:23.131654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" event={"ID":"e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa","Type":"ContainerStarted","Data":"f41322161c18fee51367bde31e3886c2b0df1ae60703c57a541a50a892cecb07"} Jul 07 21:59:23.132676 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:23.132654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" event={"ID":"d3a3a117-7d53-47c5-b30e-f9ab475065eb","Type":"ContainerStarted","Data":"fe7c44e1b63c4ad0a3a1bb80f4deb00e375662748b41c880d1f41516f7f4d316"} Jul 07 21:59:23.857796 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:23.857752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:23.857960 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:23.857911 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:23.858004 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:23.857981 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls podName:494370d8-286d-4c39-9b7c-15995a13779d nodeName:}" failed. No retries permitted until 2026-07-07 21:59:25.857965 +0000 UTC m=+44.452724134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ct22z" (UID: "494370d8-286d-4c39-9b7c-15995a13779d") : secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:23.958640 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:23.958592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:23.958640 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:23.958644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:23.958888 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:23.958771 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 21:59:23.958888 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:23.958801 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 21:59:23.958888 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:23.958843 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert podName:f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:25.958827511 +0000 UTC m=+44.553586645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vbppw" (UID: "f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c") : secret "networking-console-plugin-cert" not found Jul 07 21:59:23.958888 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:23.958868 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls podName:f940dfb9-d9e0-4847-8eef-b03b13bf73ca nodeName:}" failed. No retries permitted until 2026-07-07 21:59:25.958850418 +0000 UTC m=+44.553609553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8q9p" (UID: "f940dfb9-d9e0-4847-8eef-b03b13bf73ca") : secret "samples-operator-tls" not found Jul 07 21:59:24.138195 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:24.138105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" event={"ID":"d3a3a117-7d53-47c5-b30e-f9ab475065eb","Type":"ContainerStarted","Data":"fe388ce03d0333a8b2778c9df9dc9f242292d9f84c777315228e1a5751196899"} Jul 07 21:59:24.155756 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:24.155694 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-j7fhc" podStartSLOduration=0.730521566 podStartE2EDuration="2.155671594s" totalImagesPullingTime="1.425150028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:24.154746757 +0000 UTC m=+42.749505914" watchObservedRunningTime="2026-07-07 21:59:24.155671594 +0000 UTC m=+42.750430753" Jul 07 21:59:24.362573 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:24.362536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:24.362753 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:24.362719 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 21:59:24.362753 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:24.362749 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c79b7c7b-54rf4: secret "image-registry-tls" not found Jul 07 21:59:24.362945 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:24.362819 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls podName:f348a962-e8d6-4321-b95c-f0495c5c75bf nodeName:}" failed. No retries permitted until 2026-07-07 21:59:32.362798684 +0000 UTC m=+50.957557821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls") pod "image-registry-9c79b7c7b-54rf4" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf") : secret "image-registry-tls" not found Jul 07 21:59:24.463569 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:24.463463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:24.463732 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:24.463628 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 21:59:24.463732 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:24.463703 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls podName:24de3685-1707-4673-9b91-33499812966c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:32.463684676 +0000 UTC m=+51.058443814 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls") pod "dns-default-l4ptc" (UID: "24de3685-1707-4673-9b91-33499812966c") : secret "dns-default-metrics-tls" not found Jul 07 21:59:25.875312 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:25.875275 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:25.875719 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:25.875436 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:25.875719 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:25.875504 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls podName:494370d8-286d-4c39-9b7c-15995a13779d nodeName:}" failed. No retries permitted until 2026-07-07 21:59:29.875488318 +0000 UTC m=+48.470247454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ct22z" (UID: "494370d8-286d-4c39-9b7c-15995a13779d") : secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:25.976433 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:25.976395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:25.976706 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:25.976499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:25.976706 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:25.976552 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 21:59:25.976706 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:25.976588 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 21:59:25.976706 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:25.976630 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls podName:f940dfb9-d9e0-4847-8eef-b03b13bf73ca nodeName:}" failed. No retries permitted until 2026-07-07 21:59:29.976611311 +0000 UTC m=+48.571370472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8q9p" (UID: "f940dfb9-d9e0-4847-8eef-b03b13bf73ca") : secret "samples-operator-tls" not found Jul 07 21:59:25.976706 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:25.976648 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert podName:f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:29.976640004 +0000 UTC m=+48.571399147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vbppw" (UID: "f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c") : secret "networking-console-plugin-cert" not found Jul 07 21:59:27.146441 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:27.146402 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" event={"ID":"e7065e6f-ac04-4ee5-bb1a-b1dc3b4ba4aa","Type":"ContainerStarted","Data":"c2a9efdc81aa85f5768c54a6c83e674c26f18b682a0035fc8a622d5165b26b50"} Jul 07 21:59:27.164092 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:27.164028 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rrm5w" podStartSLOduration=1.7229400959999999 podStartE2EDuration="5.164006145s" totalImagesPullingTime="3.441066049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:27.163665211 +0000 UTC m=+45.758424367" watchObservedRunningTime="2026-07-07 21:59:27.164006145 +0000 UTC m=+45.758765283" Jul 07 21:59:28.595640 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.595603 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8"] Jul 07 21:59:28.632732 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.632691 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8"] Jul 07 21:59:28.632896 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.632838 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" Jul 07 21:59:28.638909 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.638880 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nvc2f\"" Jul 07 21:59:28.639379 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.639352 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 21:59:28.639379 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.639372 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 21:59:28.700358 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.700298 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zg9r\" (UniqueName: \"kubernetes.io/projected/a166678e-5c5b-490d-ac16-7f387a4436a0-kube-api-access-9zg9r\") pod \"migrator-d57d58cc9-gprw8\" (UID: \"a166678e-5c5b-490d-ac16-7f387a4436a0\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" Jul 07 21:59:28.801669 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.801628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zg9r\" (UniqueName: \"kubernetes.io/projected/a166678e-5c5b-490d-ac16-7f387a4436a0-kube-api-access-9zg9r\") pod \"migrator-d57d58cc9-gprw8\" (UID: \"a166678e-5c5b-490d-ac16-7f387a4436a0\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" Jul 07 21:59:28.821121 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.821083 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lk65s"] Jul 07 21:59:28.825497 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.825471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zg9r\" (UniqueName: \"kubernetes.io/projected/a166678e-5c5b-490d-ac16-7f387a4436a0-kube-api-access-9zg9r\") pod \"migrator-d57d58cc9-gprw8\" (UID: \"a166678e-5c5b-490d-ac16-7f387a4436a0\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" Jul 07 21:59:28.844567 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.844532 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lk65s"] Jul 07 21:59:28.844752 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.844652 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:28.847871 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.847800 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 21:59:28.847871 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.847812 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 21:59:28.848056 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.847872 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 21:59:28.848056 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.847896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lz8kg\"" Jul 07 21:59:28.848056 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.847808 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 21:59:28.902398 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.902360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9012fc19-7a26-46f0-8193-eac305bbf8fd-data-volume\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:28.902398 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.902402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9012fc19-7a26-46f0-8193-eac305bbf8fd-crio-socket\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:28.902654 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.902492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:28.902654 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.902512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gtr\" (UniqueName: \"kubernetes.io/projected/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-api-access-v9gtr\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:28.902753 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.902664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:28.945367 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:28.945310 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" Jul 07 21:59:29.003974 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.003924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gtr\" (UniqueName: \"kubernetes.io/projected/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-api-access-v9gtr\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.004118 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.004057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.004174 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.004126 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9012fc19-7a26-46f0-8193-eac305bbf8fd-data-volume\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.004174 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.004158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9012fc19-7a26-46f0-8193-eac305bbf8fd-crio-socket\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.004267 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.004196 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 21:59:29.004267 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.004236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.004267 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.004254 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:29.504237896 +0000 UTC m=+48.098997031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : secret "insights-runtime-extractor-tls" not found Jul 07 21:59:29.004447 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.004296 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 21:59:29.004447 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.004393 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:29.504376577 +0000 UTC m=+48.099135737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : configmap "kube-rbac-proxy" not found Jul 07 21:59:29.004552 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.004444 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9012fc19-7a26-46f0-8193-eac305bbf8fd-crio-socket\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.004613 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.004594 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9012fc19-7a26-46f0-8193-eac305bbf8fd-data-volume\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.029087 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.029055 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gtr\" (UniqueName: \"kubernetes.io/projected/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-api-access-v9gtr\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.097980 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.097902 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8"] Jul 07 21:59:29.101022 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:29.100995 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda166678e_5c5b_490d_ac16_7f387a4436a0.slice/crio-50c1883fd2d9bb2f3b91ea3d69c6288a3b34b9282a4a5514e00490d5f8c64d26 WatchSource:0}: Error finding container 50c1883fd2d9bb2f3b91ea3d69c6288a3b34b9282a4a5514e00490d5f8c64d26: Status 404 returned error can't find the container with id 50c1883fd2d9bb2f3b91ea3d69c6288a3b34b9282a4a5514e00490d5f8c64d26 Jul 07 21:59:29.152472 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.152429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" event={"ID":"a166678e-5c5b-490d-ac16-7f387a4436a0","Type":"ContainerStarted","Data":"50c1883fd2d9bb2f3b91ea3d69c6288a3b34b9282a4a5514e00490d5f8c64d26"} Jul 07 21:59:29.509113 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.509013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.509281 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.509118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:29.509281 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.509190 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 21:59:29.509281 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.509203 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 21:59:29.509281 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.509258 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:30.509240992 +0000 UTC m=+49.104000152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : secret "insights-runtime-extractor-tls" not found Jul 07 21:59:29.509281 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.509273 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:30.50926633 +0000 UTC m=+49.104025465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : configmap "kube-rbac-proxy" not found Jul 07 21:59:29.912010 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:29.911958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:29.912416 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.912118 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:29.912416 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:29.912198 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls podName:494370d8-286d-4c39-9b7c-15995a13779d nodeName:}" failed. No retries permitted until 2026-07-07 21:59:37.912181313 +0000 UTC m=+56.506940448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ct22z" (UID: "494370d8-286d-4c39-9b7c-15995a13779d") : secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:30.013115 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:30.013078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:30.013295 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:30.013150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:30.013295 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.013239 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 21:59:30.013448 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.013311 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls podName:f940dfb9-d9e0-4847-8eef-b03b13bf73ca nodeName:}" failed. No retries permitted until 2026-07-07 21:59:38.013295287 +0000 UTC m=+56.608054424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8q9p" (UID: "f940dfb9-d9e0-4847-8eef-b03b13bf73ca") : secret "samples-operator-tls" not found Jul 07 21:59:30.013448 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.013317 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 21:59:30.013448 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.013411 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert podName:f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:38.013392894 +0000 UTC m=+56.608152047 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vbppw" (UID: "f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c") : secret "networking-console-plugin-cert" not found Jul 07 21:59:30.517565 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:30.517532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:30.517735 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:30.517616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:30.517735 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.517662 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 21:59:30.517806 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.517739 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 21:59:30.517806 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.517742 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:32.517723567 +0000 UTC m=+51.112482711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : configmap "kube-rbac-proxy" not found Jul 07 21:59:30.517806 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:30.517785 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:32.517774572 +0000 UTC m=+51.112533707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : secret "insights-runtime-extractor-tls" not found Jul 07 21:59:31.159071 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.159034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" event={"ID":"a166678e-5c5b-490d-ac16-7f387a4436a0","Type":"ContainerStarted","Data":"0d02d17a607d85f2d387cb1f484c89cda2d78f3fb8602306f822330d5cffbbe0"} Jul 07 21:59:31.159601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.159091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" event={"ID":"a166678e-5c5b-490d-ac16-7f387a4436a0","Type":"ContainerStarted","Data":"036fdff5b197100f747d72d2f123f8e6b8e4d4082588aff795bc1fd8ac292ca2"} Jul 07 21:59:31.428161 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.428085 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vp854_6dcab854-eee8-4d5d-a448-602298a4e71d/dns-node-resolver/0.log" Jul 07 21:59:31.605283 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.605152 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-gprw8" podStartSLOduration=2.078748201 podStartE2EDuration="3.605131426s" totalImagesPullingTime="1.526383225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:31.17950768 +0000 UTC m=+49.774266836" watchObservedRunningTime="2026-07-07 21:59:31.605131426 +0000 UTC m=+50.199890580" Jul 07 21:59:31.606617 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.606588 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-g6rgb"] Jul 07 21:59:31.623490 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.623457 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-g6rgb"] Jul 07 21:59:31.623668 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.623561 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.626315 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.626288 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 21:59:31.626517 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.626347 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 21:59:31.626517 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.626408 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 21:59:31.627287 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.627273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xc4tg\"" Jul 07 21:59:31.627365 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.627273 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 21:59:31.729681 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.729583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-signing-cabundle\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.729681 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.729650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-499dx\" (UniqueName: \"kubernetes.io/projected/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-kube-api-access-499dx\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.729868 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.729700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-signing-key\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.830727 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.830684 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-signing-cabundle\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.830893 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.830748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-499dx\" (UniqueName: \"kubernetes.io/projected/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-kube-api-access-499dx\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.830893 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.830774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-signing-key\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.831441 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.831411 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-signing-cabundle\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.833492 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.833466 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-signing-key\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.850210 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.850181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-499dx\" (UniqueName: \"kubernetes.io/projected/ee2e9736-ad9a-4e3e-99dd-987c34c8e872-kube-api-access-499dx\") pod \"service-ca-758f599c5-g6rgb\" (UID: \"ee2e9736-ad9a-4e3e-99dd-987c34c8e872\") " pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:31.933679 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:31.933645 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-g6rgb" Jul 07 21:59:32.091855 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.091819 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-g6rgb"] Jul 07 21:59:32.094666 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:32.094638 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee2e9736_ad9a_4e3e_99dd_987c34c8e872.slice/crio-c99b932940a332552783574799dc3ce1e5b6dfcf965cb7e24f8e11d6b4c44267 WatchSource:0}: Error finding container c99b932940a332552783574799dc3ce1e5b6dfcf965cb7e24f8e11d6b4c44267: Status 404 returned error can't find the container with id c99b932940a332552783574799dc3ce1e5b6dfcf965cb7e24f8e11d6b4c44267 Jul 07 21:59:32.162442 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.162404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-g6rgb" event={"ID":"ee2e9736-ad9a-4e3e-99dd-987c34c8e872","Type":"ContainerStarted","Data":"c99b932940a332552783574799dc3ce1e5b6dfcf965cb7e24f8e11d6b4c44267"} Jul 07 21:59:32.427727 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.427698 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m9qs8_d51c526d-3e94-4c65-821c-ae2b5466a4bb/node-ca/0.log" Jul 07 21:59:32.437152 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.437120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:32.437321 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.437238 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 21:59:32.437321 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.437250 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c79b7c7b-54rf4: secret "image-registry-tls" not found Jul 07 21:59:32.437321 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.437301 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls podName:f348a962-e8d6-4321-b95c-f0495c5c75bf nodeName:}" failed. No retries permitted until 2026-07-07 21:59:48.437287599 +0000 UTC m=+67.032046734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls") pod "image-registry-9c79b7c7b-54rf4" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf") : secret "image-registry-tls" not found Jul 07 21:59:32.538281 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.538243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:32.538458 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.538362 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:32.538458 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.538399 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 21:59:32.538458 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:32.538419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:32.538603 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.538474 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:36.53845316 +0000 UTC m=+55.133212309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : configmap "kube-rbac-proxy" not found Jul 07 21:59:32.538603 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.538500 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 21:59:32.538603 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.538503 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 21:59:32.538603 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.538547 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:36.538535413 +0000 UTC m=+55.133294547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : secret "insights-runtime-extractor-tls" not found Jul 07 21:59:32.538603 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:32.538561 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls podName:24de3685-1707-4673-9b91-33499812966c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:48.538554964 +0000 UTC m=+67.133314098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls") pod "dns-default-l4ptc" (UID: "24de3685-1707-4673-9b91-33499812966c") : secret "dns-default-metrics-tls" not found Jul 07 21:59:33.029415 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:33.029382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tmvwz_259d2c92-3c9b-4475-8693-e0eb6bd3969c/serve-healthcheck-canary/0.log" Jul 07 21:59:33.229204 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:33.229164 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-gprw8_a166678e-5c5b-490d-ac16-7f387a4436a0/migrator/0.log" Jul 07 21:59:33.429409 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:33.429366 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-gprw8_a166678e-5c5b-490d-ac16-7f387a4436a0/graceful-termination/0.log" Jul 07 21:59:35.172801 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:35.172746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-g6rgb" event={"ID":"ee2e9736-ad9a-4e3e-99dd-987c34c8e872","Type":"ContainerStarted","Data":"6fa26f263f89e17f78beb958fe18c68777213fd4b10374aee7a1b8eea4c0049d"} Jul 07 21:59:35.193303 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:35.193236 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-g6rgb" podStartSLOduration=1.643914834 podStartE2EDuration="4.193212879s" totalImagesPullingTime="2.549298045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:35.192111925 +0000 UTC m=+53.786871084" watchObservedRunningTime="2026-07-07 21:59:35.193212879 +0000 UTC m=+53.787972037" Jul 07 21:59:36.574219 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:36.574170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:36.574711 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:36.574382 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 21:59:36.574711 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:36.574478 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls podName:9012fc19-7a26-46f0-8193-eac305bbf8fd nodeName:}" failed. No retries permitted until 2026-07-07 21:59:44.574456128 +0000 UTC m=+63.169215267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lk65s" (UID: "9012fc19-7a26-46f0-8193-eac305bbf8fd") : secret "insights-runtime-extractor-tls" not found Jul 07 21:59:36.574918 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:36.574894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:36.574972 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:36.574937 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9012fc19-7a26-46f0-8193-eac305bbf8fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:37.988493 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:37.988453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:37.988900 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:37.988608 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:37.988900 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:37.988678 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls podName:494370d8-286d-4c39-9b7c-15995a13779d nodeName:}" failed. No retries permitted until 2026-07-07 21:59:53.988661122 +0000 UTC m=+72.583420257 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ct22z" (UID: "494370d8-286d-4c39-9b7c-15995a13779d") : secret "cluster-monitoring-operator-tls" not found Jul 07 21:59:38.089049 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:38.089006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:38.089253 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:38.089117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:38.089253 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:38.089173 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 21:59:38.089322 ip-10-0-132-125 kubenswrapper[2562]: E0707 21:59:38.089260 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert podName:f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c nodeName:}" failed. No retries permitted until 2026-07-07 21:59:54.08924299 +0000 UTC m=+72.684002127 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vbppw" (UID: "f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c") : secret "networking-console-plugin-cert" not found Jul 07 21:59:38.091665 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:38.091637 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f940dfb9-d9e0-4847-8eef-b03b13bf73ca-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8q9p\" (UID: \"f940dfb9-d9e0-4847-8eef-b03b13bf73ca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:38.385826 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:38.385784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" Jul 07 21:59:38.550607 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:38.550576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p"] Jul 07 21:59:39.186733 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:39.186700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" event={"ID":"f940dfb9-d9e0-4847-8eef-b03b13bf73ca","Type":"ContainerStarted","Data":"f139aec021c7e6936d395655ec7f7e6197355c2ee1005f4d69efbd0ba395e5f4"} Jul 07 21:59:41.194601 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:41.194556 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" event={"ID":"f940dfb9-d9e0-4847-8eef-b03b13bf73ca","Type":"ContainerStarted","Data":"22f817a5e50e88efb33e163eb38a1209fe890a78fff32e508917cf52987fdd50"} Jul 07 21:59:41.195005 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:41.194629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" event={"ID":"f940dfb9-d9e0-4847-8eef-b03b13bf73ca","Type":"ContainerStarted","Data":"ecb75d0fbb116642580895ea810ed6eac6453ce8fde12db71236d5a5878e4459"} Jul 07 21:59:41.215560 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:41.215500 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8q9p" podStartSLOduration=17.45492364 podStartE2EDuration="19.21548436s" totalImagesPullingTime="1.76056072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:41.21353897 +0000 UTC m=+59.808298126" watchObservedRunningTime="2026-07-07 21:59:41.21548436 +0000 UTC m=+59.810243557" Jul 07 21:59:42.101091 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:42.101059 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-69lbw" Jul 07 21:59:44.651077 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:44.651034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:44.653550 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:44.653514 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9012fc19-7a26-46f0-8193-eac305bbf8fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lk65s\" (UID: \"9012fc19-7a26-46f0-8193-eac305bbf8fd\") " pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:44.756198 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:44.756157 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lz8kg\"" Jul 07 21:59:44.764062 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:44.764019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lk65s" Jul 07 21:59:44.901010 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:44.900977 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lk65s"] Jul 07 21:59:44.904432 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:44.904399 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9012fc19_7a26_46f0_8193_eac305bbf8fd.slice/crio-9e030c1d80e420d8c143e4119aec68702affab470fb9b6c61ac132a14476a9a8 WatchSource:0}: Error finding container 9e030c1d80e420d8c143e4119aec68702affab470fb9b6c61ac132a14476a9a8: Status 404 returned error can't find the container with id 9e030c1d80e420d8c143e4119aec68702affab470fb9b6c61ac132a14476a9a8 Jul 07 21:59:45.208041 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:45.207940 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lk65s" event={"ID":"9012fc19-7a26-46f0-8193-eac305bbf8fd","Type":"ContainerStarted","Data":"9e030c1d80e420d8c143e4119aec68702affab470fb9b6c61ac132a14476a9a8"} Jul 07 21:59:47.679659 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.679626 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:47.682287 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.682262 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfbn9\" (UniqueName: \"kubernetes.io/projected/5e8e2ad6-8c16-49de-a957-8bd1692fbf9c-kube-api-access-bfbn9\") pod \"network-check-target-cffq9\" (UID: \"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c\") " pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:47.780805 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.780767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:47.780994 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.780885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:47.784185 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.784155 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 21:59:47.784849 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.784826 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 21:59:47.794585 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.794538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9eba0b17-6bdb-4a42-ad6e-ba0d705772ef-original-pull-secret\") pod \"global-pull-secret-syncer-l6chz\" (UID: \"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef\") " pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:47.794739 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.794555 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a61b1873-a762-4e95-aa01-818198e15cc3-metrics-certs\") pod \"network-metrics-daemon-htffx\" (UID: \"a61b1873-a762-4e95-aa01-818198e15cc3\") " pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:47.877993 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.877950 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqsmx\"" Jul 07 21:59:47.878177 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.878017 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6chz" Jul 07 21:59:47.880243 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.880211 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b8gff\"" Jul 07 21:59:47.884045 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.884012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htffx" Jul 07 21:59:47.888560 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:47.888534 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:48.487187 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.487145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:48.490086 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.490054 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"image-registry-9c79b7c7b-54rf4\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:48.588221 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.588179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:48.591192 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.591151 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24de3685-1707-4673-9b91-33499812966c-metrics-tls\") pod \"dns-default-l4ptc\" (UID: \"24de3685-1707-4673-9b91-33499812966c\") " pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:48.720430 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.720396 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzmhm\"" Jul 07 21:59:48.722700 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.722638 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:48.726924 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.726671 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6c5lw\"" Jul 07 21:59:48.734837 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.734618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:48.780818 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.780358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cffq9"] Jul 07 21:59:48.784097 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:48.784060 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8e2ad6_8c16_49de_a957_8bd1692fbf9c.slice/crio-2d81fab07d93b7f68d3d53933c22ce6979025d2bd938d3dd8916fda4799ff1eb WatchSource:0}: Error finding container 2d81fab07d93b7f68d3d53933c22ce6979025d2bd938d3dd8916fda4799ff1eb: Status 404 returned error can't find the container with id 2d81fab07d93b7f68d3d53933c22ce6979025d2bd938d3dd8916fda4799ff1eb Jul 07 21:59:48.816077 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.813209 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l6chz"] Jul 07 21:59:48.839718 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.839648 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-htffx"] Jul 07 21:59:48.927680 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.927649 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9c79b7c7b-54rf4"] Jul 07 21:59:48.931077 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:48.931048 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf348a962_e8d6_4321_b95c_f0495c5c75bf.slice/crio-92d6587534cd115813a0373a27dce2c9f072f7e14cee89035571bd3c2bcff50d WatchSource:0}: Error finding container 92d6587534cd115813a0373a27dce2c9f072f7e14cee89035571bd3c2bcff50d: Status 404 returned error can't find the container with id 92d6587534cd115813a0373a27dce2c9f072f7e14cee89035571bd3c2bcff50d Jul 07 21:59:48.940708 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:48.940679 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l4ptc"] Jul 07 21:59:48.945129 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:48.945045 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24de3685_1707_4673_9b91_33499812966c.slice/crio-5b449e57f2f696c5b69723465e9c00b59d97a41d044d136a1b73b4c8d42903cb WatchSource:0}: Error finding container 5b449e57f2f696c5b69723465e9c00b59d97a41d044d136a1b73b4c8d42903cb: Status 404 returned error can't find the container with id 5b449e57f2f696c5b69723465e9c00b59d97a41d044d136a1b73b4c8d42903cb Jul 07 21:59:49.223757 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.223712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htffx" event={"ID":"a61b1873-a762-4e95-aa01-818198e15cc3","Type":"ContainerStarted","Data":"bc33cc97de9d05e13319d43982a8396ed8bbed935f0e65d383dc6df7728823bb"} Jul 07 21:59:49.225894 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.225861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lk65s" event={"ID":"9012fc19-7a26-46f0-8193-eac305bbf8fd","Type":"ContainerStarted","Data":"85662a04b075eacb4f4a2ab34d146db0dcf4819e086d9b898436f038326e0cb8"} Jul 07 21:59:49.227370 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.227319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l4ptc" event={"ID":"24de3685-1707-4673-9b91-33499812966c","Type":"ContainerStarted","Data":"5b449e57f2f696c5b69723465e9c00b59d97a41d044d136a1b73b4c8d42903cb"} Jul 07 21:59:49.229284 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.229259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" event={"ID":"f348a962-e8d6-4321-b95c-f0495c5c75bf","Type":"ContainerStarted","Data":"3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406"} Jul 07 21:59:49.229446 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.229299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" event={"ID":"f348a962-e8d6-4321-b95c-f0495c5c75bf","Type":"ContainerStarted","Data":"92d6587534cd115813a0373a27dce2c9f072f7e14cee89035571bd3c2bcff50d"} Jul 07 21:59:49.229446 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.229374 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 21:59:49.230725 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.230669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l6chz" event={"ID":"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef","Type":"ContainerStarted","Data":"7c262e8abc902f000afe6d05ae5acd409cb6e6b82c346a936eda2a8a037e5bd5"} Jul 07 21:59:49.232854 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.232400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cffq9" event={"ID":"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c","Type":"ContainerStarted","Data":"d2673cbd5c3a900114dfba7e3613e093391b0b013e3802ebdd373c6a567fbeea"} Jul 07 21:59:49.232854 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.232444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cffq9" event={"ID":"5e8e2ad6-8c16-49de-a957-8bd1692fbf9c","Type":"ContainerStarted","Data":"2d81fab07d93b7f68d3d53933c22ce6979025d2bd938d3dd8916fda4799ff1eb"} Jul 07 21:59:49.232854 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.232558 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 21:59:49.276057 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.275269 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cffq9" podStartSLOduration=68.275244216 podStartE2EDuration="1m8.275244216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:49.273720495 +0000 UTC m=+67.868479652" watchObservedRunningTime="2026-07-07 21:59:49.275244216 +0000 UTC m=+67.870003373" Jul 07 21:59:49.276057 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:49.275473 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" podStartSLOduration=68.275462882 podStartE2EDuration="1m8.275462882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:49.253545853 +0000 UTC m=+67.848305205" watchObservedRunningTime="2026-07-07 21:59:49.275462882 +0000 UTC m=+67.870222042" Jul 07 21:59:50.242552 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:50.242499 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lk65s" event={"ID":"9012fc19-7a26-46f0-8193-eac305bbf8fd","Type":"ContainerStarted","Data":"59a570de5e26680fdce4424cb8f1ed62b4a70e11bdd3dbd989ea855a59317040"} Jul 07 21:59:50.247168 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:50.247134 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htffx" event={"ID":"a61b1873-a762-4e95-aa01-818198e15cc3","Type":"ContainerStarted","Data":"175c0b48e578437b881d69af4bd4f7277870611725a9e36e5a04427306e0e1d4"} Jul 07 21:59:50.247363 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:50.247180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htffx" event={"ID":"a61b1873-a762-4e95-aa01-818198e15cc3","Type":"ContainerStarted","Data":"a4cdc457fe01e4ce14d236a400e65d371e3238bd67a422840dc5070cfdafce59"} Jul 07 21:59:53.672493 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.669830 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-htffx" podStartSLOduration=71.730595171 podStartE2EDuration="1m12.669805434s" totalImagesPullingTime="939.210263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:50.269587808 +0000 UTC m=+68.864346967" watchObservedRunningTime="2026-07-07 21:59:53.669805434 +0000 UTC m=+72.264564592" Jul 07 21:59:53.672493 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.670057 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82"] Jul 07 21:59:53.673120 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.672906 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.676932 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.676900 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 21:59:53.677102 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.677081 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 21:59:53.677176 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.677142 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 21:59:53.677748 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.677728 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 21:59:53.677748 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.677726 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 21:59:53.677939 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.677728 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 21:59:53.677998 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.677944 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 21:59:53.699435 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.699396 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82"] Jul 07 21:59:53.832781 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.832747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/236cf0d5-5901-40dd-8532-8a9e6aa995f2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.832980 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.832807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.832980 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.832836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqn6r\" (UniqueName: \"kubernetes.io/projected/236cf0d5-5901-40dd-8532-8a9e6aa995f2-kube-api-access-hqn6r\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.832980 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.832915 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.833191 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.832998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-ca\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.833191 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.833033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-hub\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.934043 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.933946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.934043 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.934018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-ca\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.934043 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.934046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-hub\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.934318 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.934105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/236cf0d5-5901-40dd-8532-8a9e6aa995f2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.934318 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.934138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.934318 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.934165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqn6r\" (UniqueName: \"kubernetes.io/projected/236cf0d5-5901-40dd-8532-8a9e6aa995f2-kube-api-access-hqn6r\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.935188 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.935153 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/236cf0d5-5901-40dd-8532-8a9e6aa995f2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.937068 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.937045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.937290 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.937270 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-hub\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.937613 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.937592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.938523 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.938501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/236cf0d5-5901-40dd-8532-8a9e6aa995f2-ca\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.958080 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.958045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqn6r\" (UniqueName: \"kubernetes.io/projected/236cf0d5-5901-40dd-8532-8a9e6aa995f2-kube-api-access-hqn6r\") pod \"cluster-proxy-proxy-agent-6d87fb787d-tpb82\" (UID: \"236cf0d5-5901-40dd-8532-8a9e6aa995f2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:53.998365 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:53.998309 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" Jul 07 21:59:54.035321 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.035277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:54.038445 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.038417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/494370d8-286d-4c39-9b7c-15995a13779d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ct22z\" (UID: \"494370d8-286d-4c39-9b7c-15995a13779d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:54.135856 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.135824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:54.138747 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.138721 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vbppw\" (UID: \"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:54.266807 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.266769 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lk65s" event={"ID":"9012fc19-7a26-46f0-8193-eac305bbf8fd","Type":"ContainerStarted","Data":"2d6a5b7c19f180e8224458566a9b858ed2afd484d0890ca8cae2a89751e7c219"} Jul 07 21:59:54.269056 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.268981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l4ptc" event={"ID":"24de3685-1707-4673-9b91-33499812966c","Type":"ContainerStarted","Data":"4ab7f9d283ca856c0c0a2eae9dd9e614d3a21b596ad4bf8eee4e1a701ffbe60e"} Jul 07 21:59:54.293749 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.293439 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nnsmw\"" Jul 07 21:59:54.301155 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.299968 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lk65s" podStartSLOduration=17.24094881 podStartE2EDuration="26.299945025s" totalImagesPullingTime="9.058996215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:59:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:54.293793467 +0000 UTC m=+72.888552624" watchObservedRunningTime="2026-07-07 21:59:54.299945025 +0000 UTC m=+72.894704182" Jul 07 21:59:54.301155 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.300981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" Jul 07 21:59:54.301402 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.301365 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82"] Jul 07 21:59:54.307737 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:54.307698 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod236cf0d5_5901_40dd_8532_8a9e6aa995f2.slice/crio-74aa6329fd6cb4c33c2f97daac402145b325346e3eeb8756968a9ed53590cdb1 WatchSource:0}: Error finding container 74aa6329fd6cb4c33c2f97daac402145b325346e3eeb8756968a9ed53590cdb1: Status 404 returned error can't find the container with id 74aa6329fd6cb4c33c2f97daac402145b325346e3eeb8756968a9ed53590cdb1 Jul 07 21:59:54.395994 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.395907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4wwtq\"" Jul 07 21:59:54.402242 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.402204 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" Jul 07 21:59:54.464143 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.463783 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z"] Jul 07 21:59:54.466798 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:54.466752 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494370d8_286d_4c39_9b7c_15995a13779d.slice/crio-daa5f099185b79abf440ec7424e2733ab108e8b45cb686a4b4048d40bdf932b3 WatchSource:0}: Error finding container daa5f099185b79abf440ec7424e2733ab108e8b45cb686a4b4048d40bdf932b3: Status 404 returned error can't find the container with id daa5f099185b79abf440ec7424e2733ab108e8b45cb686a4b4048d40bdf932b3 Jul 07 21:59:54.545847 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:54.545811 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-vbppw"] Jul 07 21:59:54.551349 ip-10-0-132-125 kubenswrapper[2562]: W0707 21:59:54.551300 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f8ed4f_3ffb_4dd4_9a6f_f793b373cf6c.slice/crio-4291195fc349b2f1b96180fb683b68c4376a8dfbe457ba149fbca3ca71ae9c34 WatchSource:0}: Error finding container 4291195fc349b2f1b96180fb683b68c4376a8dfbe457ba149fbca3ca71ae9c34: Status 404 returned error can't find the container with id 4291195fc349b2f1b96180fb683b68c4376a8dfbe457ba149fbca3ca71ae9c34 Jul 07 21:59:55.278075 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.278036 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l6chz" event={"ID":"9eba0b17-6bdb-4a42-ad6e-ba0d705772ef","Type":"ContainerStarted","Data":"cd210af2a0089269f1a7edade20fb21bd8557f14ad84dbf08bfc8f73152fb4bb"} Jul 07 21:59:55.280164 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.280116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" event={"ID":"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c","Type":"ContainerStarted","Data":"4291195fc349b2f1b96180fb683b68c4376a8dfbe457ba149fbca3ca71ae9c34"} Jul 07 21:59:55.283548 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.283517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" event={"ID":"236cf0d5-5901-40dd-8532-8a9e6aa995f2","Type":"ContainerStarted","Data":"74aa6329fd6cb4c33c2f97daac402145b325346e3eeb8756968a9ed53590cdb1"} Jul 07 21:59:55.284834 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.284785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" event={"ID":"494370d8-286d-4c39-9b7c-15995a13779d","Type":"ContainerStarted","Data":"daa5f099185b79abf440ec7424e2733ab108e8b45cb686a4b4048d40bdf932b3"} Jul 07 21:59:55.289838 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.289411 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l4ptc" event={"ID":"24de3685-1707-4673-9b91-33499812966c","Type":"ContainerStarted","Data":"f59e76aa9457f98aeaeef839c636277231751f5920053d1ec6405767f67a97ae"} Jul 07 21:59:55.289976 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.289842 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l4ptc" Jul 07 21:59:55.320648 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.320584 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l6chz" podStartSLOduration=68.01650975 podStartE2EDuration="1m13.320559437s" totalImagesPullingTime="5.304049687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 21:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:55.297720065 +0000 UTC m=+73.892479225" watchObservedRunningTime="2026-07-07 21:59:55.320559437 +0000 UTC m=+73.915318596" Jul 07 21:59:55.322890 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:55.322815 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l4ptc" podStartSLOduration=34.148203431 podStartE2EDuration="39.322773032s" totalImagesPullingTime="5.174569601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:55.317839029 +0000 UTC m=+73.912598202" watchObservedRunningTime="2026-07-07 21:59:55.322773032 +0000 UTC m=+73.917532191" Jul 07 21:59:59.308635 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.308595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" event={"ID":"f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c","Type":"ContainerStarted","Data":"295836c568dfa7fc73feba7f9adeb217dbc410c7f7606b243f498bed5cd5ea71"} Jul 07 21:59:59.311093 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.311060 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" event={"ID":"236cf0d5-5901-40dd-8532-8a9e6aa995f2","Type":"ContainerStarted","Data":"9e4a1f9c8816d86d3fa3700c22cc1830bae5052229dde13014bd98d48188f919"} Jul 07 21:59:59.311257 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.311113 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" event={"ID":"236cf0d5-5901-40dd-8532-8a9e6aa995f2","Type":"ContainerStarted","Data":"90f72f56fbcad29d6d2e819f97ab3d875df0e9cd1f0f945beaf506233f5400b2"} Jul 07 21:59:59.311257 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.311129 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" event={"ID":"236cf0d5-5901-40dd-8532-8a9e6aa995f2","Type":"ContainerStarted","Data":"e9c35f92b2e889d5e7f93da6aa21d50979084da37d633822b0015223a1d5db70"} Jul 07 21:59:59.312739 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.312713 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" event={"ID":"494370d8-286d-4c39-9b7c-15995a13779d","Type":"ContainerStarted","Data":"c3126e15710cf14109f0fd039d7b510f139f9c6bdb85864b7639915ddac4ccc2"} Jul 07 21:59:59.333025 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.332973 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vbppw" podStartSLOduration=33.526675166 podStartE2EDuration="37.332954752s" totalImagesPullingTime="3.806279586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:59.330069036 +0000 UTC m=+77.924828193" watchObservedRunningTime="2026-07-07 21:59:59.332954752 +0000 UTC m=+77.927713906" Jul 07 21:59:59.361438 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.361374 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d87fb787d-tpb82" podStartSLOduration=2.289260988 podStartE2EDuration="6.361354197s" totalImagesPullingTime="4.072093209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:59.360422664 +0000 UTC m=+77.955181821" watchObservedRunningTime="2026-07-07 21:59:59.361354197 +0000 UTC m=+77.956113354" Jul 07 21:59:59.385863 ip-10-0-132-125 kubenswrapper[2562]: I0707 21:59:59.385813 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ct22z" podStartSLOduration=33.489071063 podStartE2EDuration="37.385793818s" totalImagesPullingTime="3.896722755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 21:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 21:59:59.385361398 +0000 UTC m=+77.980120556" watchObservedRunningTime="2026-07-07 21:59:59.385793818 +0000 UTC m=+77.980552974" Jul 07 22:00:06.301318 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:06.301193 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l4ptc" Jul 07 22:00:08.726989 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:08.726951 2562 patch_prober.go:28] interesting pod/image-registry-9c79b7c7b-54rf4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 22:00:08.727418 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:08.727017 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" podUID="f348a962-e8d6-4321-b95c-f0495c5c75bf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 22:00:10.251723 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:10.251694 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 22:00:11.469771 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.469737 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ff69k"] Jul 07 22:00:11.480017 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.479991 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.482547 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.482522 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 22:00:11.482666 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.482632 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 22:00:11.482754 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.482677 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 22:00:11.483249 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.483232 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 22:00:11.483625 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.483609 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kt52f\"" Jul 07 22:00:11.576531 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-root\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576716 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-wtmp\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576716 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-sys\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576716 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-tls\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576716 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-accelerators-collector-config\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576944 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576944 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-metrics-client-ca\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576944 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-textfile\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.576944 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.576833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f25gw\" (UniqueName: \"kubernetes.io/projected/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-kube-api-access-f25gw\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677513 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-sys\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677513 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-tls\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677765 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-accelerators-collector-config\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677765 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-sys\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677765 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677765 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-metrics-client-ca\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677963 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-textfile\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677963 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f25gw\" (UniqueName: \"kubernetes.io/projected/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-kube-api-access-f25gw\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677963 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-root\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.677963 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.677880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-wtmp\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.678147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.678040 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-wtmp\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.678193 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.678170 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-accelerators-collector-config\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.678243 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.678194 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-root\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.678293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.678252 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-textfile\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.678293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.678281 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-metrics-client-ca\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.680043 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.680019 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-tls\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.680156 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.680141 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.686412 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.686395 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f25gw\" (UniqueName: \"kubernetes.io/projected/6a5edc49-24aa-430e-aceb-8fd4b52b2d33-kube-api-access-f25gw\") pod \"node-exporter-ff69k\" (UID: \"6a5edc49-24aa-430e-aceb-8fd4b52b2d33\") " pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.789584 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:11.789477 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ff69k" Jul 07 22:00:11.800285 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:00:11.800254 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5edc49_24aa_430e_aceb_8fd4b52b2d33.slice/crio-17835938b9b5b2351eafee7ce36244f1826d1e05dc7680adcb2ec182a5afc30e WatchSource:0}: Error finding container 17835938b9b5b2351eafee7ce36244f1826d1e05dc7680adcb2ec182a5afc30e: Status 404 returned error can't find the container with id 17835938b9b5b2351eafee7ce36244f1826d1e05dc7680adcb2ec182a5afc30e Jul 07 22:00:12.364778 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:12.364731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ff69k" event={"ID":"6a5edc49-24aa-430e-aceb-8fd4b52b2d33","Type":"ContainerStarted","Data":"17835938b9b5b2351eafee7ce36244f1826d1e05dc7680adcb2ec182a5afc30e"} Jul 07 22:00:13.370784 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:13.370752 2562 generic.go:356] "Generic (PLEG): container finished" podID="6a5edc49-24aa-430e-aceb-8fd4b52b2d33" containerID="401807aab50e02b2eeaa0a375f7c05c7291458ee822faa67e991f9da77d0e696" exitCode=0 Jul 07 22:00:13.371185 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:13.370820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ff69k" event={"ID":"6a5edc49-24aa-430e-aceb-8fd4b52b2d33","Type":"ContainerDied","Data":"401807aab50e02b2eeaa0a375f7c05c7291458ee822faa67e991f9da77d0e696"} Jul 07 22:00:14.377290 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:14.377246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ff69k" event={"ID":"6a5edc49-24aa-430e-aceb-8fd4b52b2d33","Type":"ContainerStarted","Data":"1d1737c260e93deeca27cfe621e096d5b581fbb90a05f746901cece0a4ff166e"} Jul 07 22:00:14.377766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:14.377301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ff69k" event={"ID":"6a5edc49-24aa-430e-aceb-8fd4b52b2d33","Type":"ContainerStarted","Data":"95bdf7ead5043d79c2405b808720b0ce8dd81be9d16102893f8d4da7ef85462d"} Jul 07 22:00:14.401525 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:14.401382 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ff69k" podStartSLOduration=2.368530884 podStartE2EDuration="3.401361252s" totalImagesPullingTime="957.718976ms" totalInitContainerRuntime="75.111392ms" isStatefulPod=true podCreationTimestamp="2026-07-07 22:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:00:14.400359974 +0000 UTC m=+92.995119134" watchObservedRunningTime="2026-07-07 22:00:14.401361252 +0000 UTC m=+92.996120406" Jul 07 22:00:18.788207 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:18.788164 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9c79b7c7b-54rf4"] Jul 07 22:00:20.249218 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:20.249185 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cffq9" Jul 07 22:00:43.809246 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:43.809179 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" podUID="f348a962-e8d6-4321-b95c-f0495c5c75bf" containerName="registry" containerID="cri-o://3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406" gracePeriod=30 Jul 07 22:00:44.063833 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.063762 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 22:00:44.126192 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126147 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-bound-sa-token\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126192 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126200 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f348a962-e8d6-4321-b95c-f0495c5c75bf-ca-trust-extracted\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126489 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126224 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126489 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126241 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-installation-pull-secrets\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126489 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126276 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-image-registry-private-configuration\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126489 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126302 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-trusted-ca\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126489 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126352 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smvpl\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-kube-api-access-smvpl\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126489 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126388 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-certificates\") pod \"f348a962-e8d6-4321-b95c-f0495c5c75bf\" (UID: \"f348a962-e8d6-4321-b95c-f0495c5c75bf\") " Jul 07 22:00:44.126882 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126821 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:00:44.127035 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.126951 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:00:44.129204 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.129135 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:00:44.129204 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.129152 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-kube-api-access-smvpl" (OuterVolumeSpecName: "kube-api-access-smvpl") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "kube-api-access-smvpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:00:44.129455 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.129262 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:00:44.129455 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.129342 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:00:44.129455 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.129398 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:00:44.135705 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.135659 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f348a962-e8d6-4321-b95c-f0495c5c75bf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f348a962-e8d6-4321-b95c-f0495c5c75bf" (UID: "f348a962-e8d6-4321-b95c-f0495c5c75bf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:00:44.227830 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227778 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smvpl\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-kube-api-access-smvpl\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.227830 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227825 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-certificates\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.227830 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227836 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-bound-sa-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.227830 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227844 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f348a962-e8d6-4321-b95c-f0495c5c75bf-ca-trust-extracted\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.227830 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227854 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f348a962-e8d6-4321-b95c-f0495c5c75bf-registry-tls\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.228131 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227862 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-installation-pull-secrets\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.228131 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227872 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f348a962-e8d6-4321-b95c-f0495c5c75bf-image-registry-private-configuration\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.228131 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.227881 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f348a962-e8d6-4321-b95c-f0495c5c75bf-trusted-ca\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:00:44.485457 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.485411 2562 generic.go:356] "Generic (PLEG): container finished" podID="f348a962-e8d6-4321-b95c-f0495c5c75bf" containerID="3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406" exitCode=0 Jul 07 22:00:44.485629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.485461 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" Jul 07 22:00:44.485629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.485488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" event={"ID":"f348a962-e8d6-4321-b95c-f0495c5c75bf","Type":"ContainerDied","Data":"3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406"} Jul 07 22:00:44.485629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.485548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9c79b7c7b-54rf4" event={"ID":"f348a962-e8d6-4321-b95c-f0495c5c75bf","Type":"ContainerDied","Data":"92d6587534cd115813a0373a27dce2c9f072f7e14cee89035571bd3c2bcff50d"} Jul 07 22:00:44.485629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.485575 2562 scope.go:117] "RemoveContainer" containerID="3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406" Jul 07 22:00:44.494620 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.494598 2562 scope.go:117] "RemoveContainer" containerID="3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406" Jul 07 22:00:44.494951 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:00:44.494927 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406\": container with ID starting with 3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406 not found: ID does not exist" containerID="3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406" Jul 07 22:00:44.495025 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.494958 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406"} err="failed to get container status \"3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406\": rpc error: code = NotFound desc = could not find container \"3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406\": container with ID starting with 3abefe975678f37b50f83d825b2d430dc780ac83182de9de21b684e22c855406 not found: ID does not exist" Jul 07 22:00:44.507111 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.507080 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9c79b7c7b-54rf4"] Jul 07 22:00:44.512120 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:44.512091 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9c79b7c7b-54rf4"] Jul 07 22:00:45.954251 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:00:45.954206 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f348a962-e8d6-4321-b95c-f0495c5c75bf" path="/var/lib/kubelet/pods/f348a962-e8d6-4321-b95c-f0495c5c75bf/volumes" Jul 07 22:03:00.745359 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.745311 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sd4th"] Jul 07 22:03:00.745841 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.745648 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f348a962-e8d6-4321-b95c-f0495c5c75bf" containerName="registry" Jul 07 22:03:00.745841 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.745667 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f348a962-e8d6-4321-b95c-f0495c5c75bf" containerName="registry" Jul 07 22:03:00.745841 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.745723 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f348a962-e8d6-4321-b95c-f0495c5c75bf" containerName="registry" Jul 07 22:03:00.748653 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.748636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:00.751046 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.751023 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 22:03:00.752007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.751988 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 22:03:00.752093 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.752079 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vmptp\"" Jul 07 22:03:00.756944 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.756917 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sd4th"] Jul 07 22:03:00.827430 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.827378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33e37b78-e34f-45e0-adf8-a3482d349db6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sd4th\" (UID: \"33e37b78-e34f-45e0-adf8-a3482d349db6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:00.827430 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.827428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdwld\" (UniqueName: \"kubernetes.io/projected/33e37b78-e34f-45e0-adf8-a3482d349db6-kube-api-access-qdwld\") pod \"cert-manager-cainjector-559cdd8c87-sd4th\" (UID: \"33e37b78-e34f-45e0-adf8-a3482d349db6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:00.928668 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.928631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33e37b78-e34f-45e0-adf8-a3482d349db6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sd4th\" (UID: \"33e37b78-e34f-45e0-adf8-a3482d349db6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:00.928668 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.928671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwld\" (UniqueName: \"kubernetes.io/projected/33e37b78-e34f-45e0-adf8-a3482d349db6-kube-api-access-qdwld\") pod \"cert-manager-cainjector-559cdd8c87-sd4th\" (UID: \"33e37b78-e34f-45e0-adf8-a3482d349db6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:00.937773 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.937739 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33e37b78-e34f-45e0-adf8-a3482d349db6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sd4th\" (UID: \"33e37b78-e34f-45e0-adf8-a3482d349db6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:00.937926 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:00.937817 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdwld\" (UniqueName: \"kubernetes.io/projected/33e37b78-e34f-45e0-adf8-a3482d349db6-kube-api-access-qdwld\") pod \"cert-manager-cainjector-559cdd8c87-sd4th\" (UID: \"33e37b78-e34f-45e0-adf8-a3482d349db6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:01.058243 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:01.058152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" Jul 07 22:03:01.214463 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:01.214425 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sd4th"] Jul 07 22:03:01.218690 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:03:01.218663 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33e37b78_e34f_45e0_adf8_a3482d349db6.slice/crio-7bc16def801fc94828c8e6b8ff3d60b8655165f52fa2a939915ac06b2f8ac51f WatchSource:0}: Error finding container 7bc16def801fc94828c8e6b8ff3d60b8655165f52fa2a939915ac06b2f8ac51f: Status 404 returned error can't find the container with id 7bc16def801fc94828c8e6b8ff3d60b8655165f52fa2a939915ac06b2f8ac51f Jul 07 22:03:01.969278 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:01.969215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" event={"ID":"33e37b78-e34f-45e0-adf8-a3482d349db6","Type":"ContainerStarted","Data":"7bc16def801fc94828c8e6b8ff3d60b8655165f52fa2a939915ac06b2f8ac51f"} Jul 07 22:03:04.984754 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:04.984718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" event={"ID":"33e37b78-e34f-45e0-adf8-a3482d349db6","Type":"ContainerStarted","Data":"88b10e1f1cc1a56b12a5cd4ab19516e89dee81ae8bffca653a45c39907344943"} Jul 07 22:03:05.002349 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:05.002286 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sd4th" podStartSLOduration=1.893008523 podStartE2EDuration="5.002269299s" totalImagesPullingTime="3.109260776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:05.001122564 +0000 UTC m=+263.595881742" watchObservedRunningTime="2026-07-07 22:03:05.002269299 +0000 UTC m=+263.597028455" Jul 07 22:03:30.498093 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.498051 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w"] Jul 07 22:03:30.502861 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.502839 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.517192 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.517162 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 22:03:30.518345 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.518311 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 22:03:30.518494 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.518454 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 22:03:30.522711 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.522690 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mn624\"" Jul 07 22:03:30.531099 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.531071 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:03:30.531242 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.531203 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 22:03:30.539649 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.539611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4pgs\" (UniqueName: \"kubernetes.io/projected/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-kube-api-access-p4pgs\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.539846 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.539799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-cert\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.539846 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.539829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-metrics-cert\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.539966 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.539877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-manager-config\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.544838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.544811 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w"] Jul 07 22:03:30.640461 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.640410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-manager-config\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.640670 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.640487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4pgs\" (UniqueName: \"kubernetes.io/projected/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-kube-api-access-p4pgs\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.640670 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.640523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-cert\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.640670 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.640540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-metrics-cert\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.641093 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.641066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-manager-config\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.643144 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.643119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-metrics-cert\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.643259 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.643243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-cert\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.667696 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.667663 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4pgs\" (UniqueName: \"kubernetes.io/projected/f4cb09eb-37d8-4fca-af8d-cf44fbd03e87-kube-api-access-p4pgs\") pod \"lws-controller-manager-5c4f8f67f4-9xc9w\" (UID: \"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87\") " pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.811400 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.811278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:30.950936 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:30.950815 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w"] Jul 07 22:03:30.953557 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:03:30.953530 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cb09eb_37d8_4fca_af8d_cf44fbd03e87.slice/crio-16617312cf9bd44cbc23690d3e446f7b8a042795d082ec2bfbc6b3a77100ccc0 WatchSource:0}: Error finding container 16617312cf9bd44cbc23690d3e446f7b8a042795d082ec2bfbc6b3a77100ccc0: Status 404 returned error can't find the container with id 16617312cf9bd44cbc23690d3e446f7b8a042795d082ec2bfbc6b3a77100ccc0 Jul 07 22:03:31.077080 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.077041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" event={"ID":"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87","Type":"ContainerStarted","Data":"16617312cf9bd44cbc23690d3e446f7b8a042795d082ec2bfbc6b3a77100ccc0"} Jul 07 22:03:31.787698 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.787346 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx"] Jul 07 22:03:31.791311 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.791282 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.803828 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.803198 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 22:03:31.803828 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.803554 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 22:03:31.803828 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.803756 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 22:03:31.804496 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.804011 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m6z9x\"" Jul 07 22:03:31.804496 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.804068 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 22:03:31.826485 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.826450 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx"] Jul 07 22:03:31.851700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.851657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87033ff9-b40d-452e-945a-07c288ac9c05-tmp\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.851906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.851730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/87033ff9-b40d-452e-945a-07c288ac9c05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.851906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.851813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/87033ff9-b40d-452e-945a-07c288ac9c05-manifests\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.851906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.851859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87033ff9-b40d-452e-945a-07c288ac9c05-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.851906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.851889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxp7\" (UniqueName: \"kubernetes.io/projected/87033ff9-b40d-452e-945a-07c288ac9c05-kube-api-access-wwxp7\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.952400 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.952322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87033ff9-b40d-452e-945a-07c288ac9c05-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.952400 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.952395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxp7\" (UniqueName: \"kubernetes.io/projected/87033ff9-b40d-452e-945a-07c288ac9c05-kube-api-access-wwxp7\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.952633 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.952429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87033ff9-b40d-452e-945a-07c288ac9c05-tmp\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.952633 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.952470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/87033ff9-b40d-452e-945a-07c288ac9c05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.952633 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.952518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/87033ff9-b40d-452e-945a-07c288ac9c05-manifests\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.953137 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.952973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/87033ff9-b40d-452e-945a-07c288ac9c05-manifests\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.956075 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.955983 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87033ff9-b40d-452e-945a-07c288ac9c05-tmp\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.956075 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.956030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/87033ff9-b40d-452e-945a-07c288ac9c05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.956274 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.956167 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87033ff9-b40d-452e-945a-07c288ac9c05-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:31.965293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:31.965257 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxp7\" (UniqueName: \"kubernetes.io/projected/87033ff9-b40d-452e-945a-07c288ac9c05-kube-api-access-wwxp7\") pod \"opendatahub-operator-controller-manager-6dd55458bc-87cpx\" (UID: \"87033ff9-b40d-452e-945a-07c288ac9c05\") " pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:32.110780 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:32.110711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:32.310876 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:32.310836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx"] Jul 07 22:03:32.314091 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:03:32.314055 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87033ff9_b40d_452e_945a_07c288ac9c05.slice/crio-c1a2b42c9a337a199b12d082377f1ce43e031f71fcd0a77a647e76e008c98524 WatchSource:0}: Error finding container c1a2b42c9a337a199b12d082377f1ce43e031f71fcd0a77a647e76e008c98524: Status 404 returned error can't find the container with id c1a2b42c9a337a199b12d082377f1ce43e031f71fcd0a77a647e76e008c98524 Jul 07 22:03:33.088749 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:33.088697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" event={"ID":"87033ff9-b40d-452e-945a-07c288ac9c05","Type":"ContainerStarted","Data":"c1a2b42c9a337a199b12d082377f1ce43e031f71fcd0a77a647e76e008c98524"} Jul 07 22:03:34.095789 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:34.095748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" event={"ID":"f4cb09eb-37d8-4fca-af8d-cf44fbd03e87","Type":"ContainerStarted","Data":"fe24c0fb5ce039eeb2c77965c8449df4e07a63d8f63d7385800dd5e600e7eb47"} Jul 07 22:03:34.096280 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:34.095862 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:34.116664 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:34.116607 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" podStartSLOduration=1.525412462 podStartE2EDuration="4.116588481s" totalImagesPullingTime="2.591176019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:34.11405611 +0000 UTC m=+292.708815270" watchObservedRunningTime="2026-07-07 22:03:34.116588481 +0000 UTC m=+292.711347637" Jul 07 22:03:36.106215 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:36.106180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" event={"ID":"87033ff9-b40d-452e-945a-07c288ac9c05","Type":"ContainerStarted","Data":"0a6a454b02b463185a852061e5ce57432d0596b4c7d25d13704ddb11328b1774"} Jul 07 22:03:38.116482 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:38.116446 2562 generic.go:356] "Generic (PLEG): container finished" podID="87033ff9-b40d-452e-945a-07c288ac9c05" containerID="0a6a454b02b463185a852061e5ce57432d0596b4c7d25d13704ddb11328b1774" exitCode=0 Jul 07 22:03:38.116482 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:38.116486 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" event={"ID":"87033ff9-b40d-452e-945a-07c288ac9c05","Type":"ContainerDied","Data":"0a6a454b02b463185a852061e5ce57432d0596b4c7d25d13704ddb11328b1774"} Jul 07 22:03:39.121983 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:39.121945 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" event={"ID":"87033ff9-b40d-452e-945a-07c288ac9c05","Type":"ContainerStarted","Data":"4f636cb32ea2618fd2de0d723645c8accb5802ce3c1a7f46edf16fac7bddcfee"} Jul 07 22:03:39.122508 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:39.122272 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:03:39.148817 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:39.148753 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" podStartSLOduration=3.05308999 podStartE2EDuration="8.148733945s" totalImagesPullingTime="4.013537286s" totalInitContainerRuntime="1.082106669s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:39.14623477 +0000 UTC m=+297.740993929" watchObservedRunningTime="2026-07-07 22:03:39.148733945 +0000 UTC m=+297.743493102" Jul 07 22:03:41.833735 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:41.833705 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 22:03:45.103373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:45.103309 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c4f8f67f4-9xc9w" Jul 07 22:03:50.127624 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:03:50.127594 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dd55458bc-87cpx" Jul 07 22:04:00.685499 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.685460 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r"] Jul 07 22:04:00.695000 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.694967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.698013 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.697986 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:04:00.698322 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.698305 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 22:04:00.698493 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.698478 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 22:04:00.699082 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.699065 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qm628\"" Jul 07 22:04:00.710575 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.710540 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r"] Jul 07 22:04:00.784528 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784528 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb36678c-afb0-4022-bbcf-38970b987150-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb36678c-afb0-4022-bbcf-38970b987150-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c87xv\" (UniqueName: \"kubernetes.io/projected/cb36678c-afb0-4022-bbcf-38970b987150-kube-api-access-c87xv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.784978 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.784790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb36678c-afb0-4022-bbcf-38970b987150-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885428 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885428 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885433 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb36678c-afb0-4022-bbcf-38970b987150-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb36678c-afb0-4022-bbcf-38970b987150-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c87xv\" (UniqueName: \"kubernetes.io/projected/cb36678c-afb0-4022-bbcf-38970b987150-kube-api-access-c87xv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.885700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb36678c-afb0-4022-bbcf-38970b987150-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.886005 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885892 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.886005 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.885903 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.886130 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.886108 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.886184 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.886140 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.886373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.886353 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb36678c-afb0-4022-bbcf-38970b987150-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.888107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.888083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb36678c-afb0-4022-bbcf-38970b987150-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.888107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.888097 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb36678c-afb0-4022-bbcf-38970b987150-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.896793 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.896765 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb36678c-afb0-4022-bbcf-38970b987150-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:00.906855 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:00.906824 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c87xv\" (UniqueName: \"kubernetes.io/projected/cb36678c-afb0-4022-bbcf-38970b987150-kube-api-access-c87xv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgk2r\" (UID: \"cb36678c-afb0-4022-bbcf-38970b987150\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:01.005899 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.005799 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:01.156906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.156808 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r"] Jul 07 22:04:01.159641 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:04:01.159614 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb36678c_afb0_4022_bbcf_38970b987150.slice/crio-997a709528677495a099fa05e92aa809f7d26dfed0deb367f1e29bd346b8bf5e WatchSource:0}: Error finding container 997a709528677495a099fa05e92aa809f7d26dfed0deb367f1e29bd346b8bf5e: Status 404 returned error can't find the container with id 997a709528677495a099fa05e92aa809f7d26dfed0deb367f1e29bd346b8bf5e Jul 07 22:04:01.161562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.161546 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:04:01.208259 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.208216 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" event={"ID":"cb36678c-afb0-4022-bbcf-38970b987150","Type":"ContainerStarted","Data":"997a709528677495a099fa05e92aa809f7d26dfed0deb367f1e29bd346b8bf5e"} Jul 07 22:04:01.937464 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.937427 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn"] Jul 07 22:04:01.940321 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.940302 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.946533 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.946315 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 22:04:01.946533 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.946314 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 22:04:01.946533 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.946339 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w6bjz\"" Jul 07 22:04:01.946815 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.946799 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 22:04:01.955020 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.954989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn"] Jul 07 22:04:01.999346 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999283 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.999536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999365 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.999536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1e4480-a950-4536-a8f0-e60886b92771-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.999536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb1e4480-a950-4536-a8f0-e60886b92771-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.999536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999498 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j2hk\" (UniqueName: \"kubernetes.io/projected/bb1e4480-a950-4536-a8f0-e60886b92771-kube-api-access-8j2hk\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.999666 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:01.999666 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:01.999592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1e4480-a950-4536-a8f0-e60886b92771-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.100696 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.100659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1e4480-a950-4536-a8f0-e60886b92771-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.100696 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.100714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.100954 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.100854 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.100954 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.100896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1e4480-a950-4536-a8f0-e60886b92771-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.101060 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.100951 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb1e4480-a950-4536-a8f0-e60886b92771-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.101060 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.101004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j2hk\" (UniqueName: \"kubernetes.io/projected/bb1e4480-a950-4536-a8f0-e60886b92771-kube-api-access-8j2hk\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.101166 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.101063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.101688 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.101662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bb1e4480-a950-4536-a8f0-e60886b92771-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.103398 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.103369 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb1e4480-a950-4536-a8f0-e60886b92771-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.103398 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.103380 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.103577 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.103454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.103634 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.103594 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bb1e4480-a950-4536-a8f0-e60886b92771-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.113057 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.113014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb1e4480-a950-4536-a8f0-e60886b92771-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.113201 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.113022 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j2hk\" (UniqueName: \"kubernetes.io/projected/bb1e4480-a950-4536-a8f0-e60886b92771-kube-api-access-8j2hk\") pod \"istiod-openshift-gateway-94bb8fbfd-n7jhn\" (UID: \"bb1e4480-a950-4536-a8f0-e60886b92771\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.251987 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.251892 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:02.399132 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:02.399100 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn"] Jul 07 22:04:02.402237 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:04:02.402204 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1e4480_a950_4536_a8f0_e60886b92771.slice/crio-a09a26e23e124e5371a063bc25e164ea940f5385db976a2937eb1731693aee1d WatchSource:0}: Error finding container a09a26e23e124e5371a063bc25e164ea940f5385db976a2937eb1731693aee1d: Status 404 returned error can't find the container with id a09a26e23e124e5371a063bc25e164ea940f5385db976a2937eb1731693aee1d Jul 07 22:04:03.218757 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:03.218716 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" event={"ID":"bb1e4480-a950-4536-a8f0-e60886b92771","Type":"ContainerStarted","Data":"a09a26e23e124e5371a063bc25e164ea940f5385db976a2937eb1731693aee1d"} Jul 07 22:04:05.767742 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:05.767701 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:04:05.768139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:05.767792 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:04:05.768139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:05.767839 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:04:06.233976 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:06.233935 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" event={"ID":"cb36678c-afb0-4022-bbcf-38970b987150","Type":"ContainerStarted","Data":"e6300a2333b6b8d3df4e4c2288e9dc3fb8287048159fdd1b1189e44ec7ab162f"} Jul 07 22:04:06.262021 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:06.261968 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" podStartSLOduration=1.656194042 podStartE2EDuration="6.261951442s" totalImagesPullingTime="4.6057574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:06.260305796 +0000 UTC m=+324.855064956" watchObservedRunningTime="2026-07-07 22:04:06.261951442 +0000 UTC m=+324.856710711" Jul 07 22:04:07.006419 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:07.006370 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:07.011395 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:07.011366 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:07.237686 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:07.237653 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:07.238895 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:07.238866 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgk2r" Jul 07 22:04:19.337948 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:19.337910 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:04:19.338211 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:19.337993 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:04:20.293182 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:20.293143 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" event={"ID":"bb1e4480-a950-4536-a8f0-e60886b92771","Type":"ContainerStarted","Data":"6d930516db13984aa03f8457b13f4683368fe178726ca4aabc4021e545737aea"} Jul 07 22:04:20.293413 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:20.293358 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:20.295080 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:20.295051 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" Jul 07 22:04:20.317653 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:04:20.317601 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n7jhn" podStartSLOduration=2.384225632 podStartE2EDuration="19.317581604s" totalImagesPullingTime="16.933355972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:20.315645183 +0000 UTC m=+338.910404339" watchObservedRunningTime="2026-07-07 22:04:20.317581604 +0000 UTC m=+338.912340761" Jul 07 22:06:52.664275 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.664237 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:06:52.666283 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.666265 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.668881 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.668841 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tb8gm\"" Jul 07 22:06:52.669006 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.668966 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 22:06:52.669678 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.669665 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 22:06:52.671012 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.670996 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 22:06:52.678516 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.678482 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:06:52.714572 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.714535 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:06:52.790468 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.790427 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c-config-file\") pod \"limitador-limitador-54d6d988d5-whwx8\" (UID: \"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.790649 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.790503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pc9g\" (UniqueName: \"kubernetes.io/projected/1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c-kube-api-access-2pc9g\") pod \"limitador-limitador-54d6d988d5-whwx8\" (UID: \"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.891037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.890985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc9g\" (UniqueName: \"kubernetes.io/projected/1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c-kube-api-access-2pc9g\") pod \"limitador-limitador-54d6d988d5-whwx8\" (UID: \"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.891195 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.891060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c-config-file\") pod \"limitador-limitador-54d6d988d5-whwx8\" (UID: \"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.891694 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.891676 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c-config-file\") pod \"limitador-limitador-54d6d988d5-whwx8\" (UID: \"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.901384 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.901359 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pc9g\" (UniqueName: \"kubernetes.io/projected/1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c-kube-api-access-2pc9g\") pod \"limitador-limitador-54d6d988d5-whwx8\" (UID: \"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:52.977320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:52.977215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:53.121945 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:53.121888 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:06:53.124310 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:06:53.124260 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7dbe0d_88aa_4cbf_bab1_f5200b2b328c.slice/crio-04baa2931d03bc0255b844563fe945b2f1ab6a7ae90613fe8bd8c940b61ad062 WatchSource:0}: Error finding container 04baa2931d03bc0255b844563fe945b2f1ab6a7ae90613fe8bd8c940b61ad062: Status 404 returned error can't find the container with id 04baa2931d03bc0255b844563fe945b2f1ab6a7ae90613fe8bd8c940b61ad062 Jul 07 22:06:53.935920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:53.935859 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" event={"ID":"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c","Type":"ContainerStarted","Data":"04baa2931d03bc0255b844563fe945b2f1ab6a7ae90613fe8bd8c940b61ad062"} Jul 07 22:06:55.946577 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:55.946462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" event={"ID":"1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c","Type":"ContainerStarted","Data":"f09f6c3bb9b11df8510a73d72cf31ae8b8c4f18636b9150c612856c7b0af7a5f"} Jul 07 22:06:55.946949 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:55.946579 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:06:55.964982 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:06:55.964927 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" podStartSLOduration=1.40456508 podStartE2EDuration="3.964906993s" totalImagesPullingTime="2.560341913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:06:55.963768343 +0000 UTC m=+494.558527502" watchObservedRunningTime="2026-07-07 22:06:55.964906993 +0000 UTC m=+494.559666171" Jul 07 22:07:06.952635 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:07:06.952600 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-whwx8" Jul 07 22:08:07.739766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:08:07.739667 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:08:29.992557 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:08:29.992517 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:08:49.906273 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:08:49.906235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:09:11.074544 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:11.074459 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:09:20.573667 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:20.573626 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:09:25.199554 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.199514 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts"] Jul 07 22:09:25.202962 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.202939 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.206265 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.206235 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vjtr8\"" Jul 07 22:09:25.206926 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.206906 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 22:09:25.207037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.206918 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 22:09:25.207037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.206984 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 22:09:25.216100 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.216067 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts"] Jul 07 22:09:25.363726 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.363683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.363726 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.363729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31f693a8-0493-4dfd-8629-5a9d7fde6021-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.363952 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.363758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.363952 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.363827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.363952 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.363859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.363952 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.363895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4275b\" (UniqueName: \"kubernetes.io/projected/31f693a8-0493-4dfd-8629-5a9d7fde6021-kube-api-access-4275b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.464854 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.464749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.464854 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.464815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31f693a8-0493-4dfd-8629-5a9d7fde6021-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.464854 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.464851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.465126 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.464887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.465126 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.464915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.465126 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.464963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4275b\" (UniqueName: \"kubernetes.io/projected/31f693a8-0493-4dfd-8629-5a9d7fde6021-kube-api-access-4275b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.465281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.465184 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.465386 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.465364 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.465444 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.465386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.467320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.467300 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31f693a8-0493-4dfd-8629-5a9d7fde6021-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.467612 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.467593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31f693a8-0493-4dfd-8629-5a9d7fde6021-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.475942 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.475905 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4275b\" (UniqueName: \"kubernetes.io/projected/31f693a8-0493-4dfd-8629-5a9d7fde6021-kube-api-access-4275b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lr9ts\" (UID: \"31f693a8-0493-4dfd-8629-5a9d7fde6021\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.513212 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.513171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:25.654554 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.654530 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts"] Jul 07 22:09:25.657169 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:09:25.657144 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f693a8_0493_4dfd_8629_5a9d7fde6021.slice/crio-fbaf65429d87cfc6052c7d21468284678a9d5b2d2148fb2d431fd7bbd0e6073f WatchSource:0}: Error finding container fbaf65429d87cfc6052c7d21468284678a9d5b2d2148fb2d431fd7bbd0e6073f: Status 404 returned error can't find the container with id fbaf65429d87cfc6052c7d21468284678a9d5b2d2148fb2d431fd7bbd0e6073f Jul 07 22:09:25.658409 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:25.658391 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:09:26.567870 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:26.567829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" event={"ID":"31f693a8-0493-4dfd-8629-5a9d7fde6021","Type":"ContainerStarted","Data":"fbaf65429d87cfc6052c7d21468284678a9d5b2d2148fb2d431fd7bbd0e6073f"} Jul 07 22:09:27.094573 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:27.094539 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:09:27.573443 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:27.573406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" event={"ID":"31f693a8-0493-4dfd-8629-5a9d7fde6021","Type":"ContainerStarted","Data":"8f69c35b72aa10cc7e9e33fec6967b2c8ecb64e68a5129c2c8e4f4c629edf091"} Jul 07 22:09:27.598944 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:27.598884 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" podStartSLOduration=1.5578544239999998 podStartE2EDuration="2.598867487s" totalImagesPullingTime="1.041013063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:09:27.597259171 +0000 UTC m=+646.192018331" watchObservedRunningTime="2026-07-07 22:09:27.598867487 +0000 UTC m=+646.193626645" Jul 07 22:09:34.780184 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:34.780145 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:09:35.513485 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:35.513438 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:35.513649 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:35.513535 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:35.526029 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:35.526001 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:09:35.619052 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:09:35.619016 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lr9ts" Jul 07 22:11:15.074315 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:11:15.074274 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:11:24.366215 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:11:24.366178 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:11:36.470428 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:11:36.470364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:11:44.463679 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:11:44.463645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:11:55.370151 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:11:55.370108 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:12:25.568569 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:12:25.568532 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:12:38.165477 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:12:38.165440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:13:13.861443 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:13:13.861404 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:13:23.072764 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:13:23.072719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:13:33.569248 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:13:33.569150 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:13:39.663161 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:13:39.663121 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:13:50.262182 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:13:50.262141 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:13:59.467788 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:13:59.467753 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:14:10.959465 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:14:10.959430 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:14:20.369698 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:14:20.369650 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:14:43.163659 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:14:43.163622 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:14:55.159992 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:14:55.159948 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:15:19.266188 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:15:19.266152 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:15:29.567412 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:15:29.567372 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:15:44.474695 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:15:44.474659 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:15:55.264443 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:15:55.264406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:16:05.160320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:16:05.160278 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:16:14.764592 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:16:14.764562 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:16:24.362638 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:16:24.362600 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:18:14.562551 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:18:14.562514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:18:23.564922 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:18:23.564882 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:18:36.863019 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:18:36.862973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:18:48.367174 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:18:48.367136 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:19:00.468871 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:19:00.468829 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:19:10.263168 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:19:10.263134 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:19:19.367409 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:19:19.367373 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:19:30.458932 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:19:30.458893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:19:41.663721 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:19:41.663640 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:19:50.566773 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:19:50.566731 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:20:01.772449 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:20:01.772410 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:20:26.057317 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:20:26.057277 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:20:33.659102 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:20:33.659068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:20:43.565915 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:20:43.565876 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:20:53.263058 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:20:53.263019 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:21:02.358508 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:21:02.358463 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:21:14.958253 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:21:14.958217 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:21:24.565684 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:21:24.565649 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:21:34.264574 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:21:34.264533 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:21:45.066811 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:21:45.066772 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:21:54.672279 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:21:54.672238 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:22:04.462609 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:22:04.462570 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:22:18.359937 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:22:18.359899 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:22:26.865352 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:22:26.865296 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:22:36.360848 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:22:36.360752 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:22:46.269675 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:22:46.269636 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:22:59.472500 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:22:59.472455 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:23:08.870268 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:23:08.870231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:23:22.058401 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:23:22.058363 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:23:33.167046 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:23:33.167007 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:23:46.462283 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:23:46.462246 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:23:59.561576 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:23:59.561536 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:24:08.266127 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:08.266038 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:24:20.172060 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:20.172023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:24:25.823442 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.823411 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg"] Jul 07 22:24:25.826838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.826814 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.831433 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.829502 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a2c4a017-openshift-default-dockercfg-6qf5j\"" Jul 07 22:24:25.847216 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.847180 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg"] Jul 07 22:24:25.965880 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.965849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-certs\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.965880 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.965891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-socket\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.965918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-envoy\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.966018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-istio-token\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.966059 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-credential-socket\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.966078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ld9b\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-kube-api-access-5ld9b\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.966120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34009265-e234-49a1-85b7-9b95440899b2-istiod-ca-cert\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966355 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.966176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-data\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:25.966355 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:25.966223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34009265-e234-49a1-85b7-9b95440899b2-istio-podinfo\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067097 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-certs\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067097 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-socket\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067097 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-envoy\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067461 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-istio-token\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067461 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067321 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-credential-socket\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067461 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ld9b\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-kube-api-access-5ld9b\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067614 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34009265-e234-49a1-85b7-9b95440899b2-istiod-ca-cert\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067614 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-data\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067614 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067571 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34009265-e234-49a1-85b7-9b95440899b2-istio-podinfo\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067614 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067580 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-certs\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067773 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067640 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-socket\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067831 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067790 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-credential-socket\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.067949 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.067923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-data\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.068130 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.068110 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34009265-e234-49a1-85b7-9b95440899b2-istiod-ca-cert\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.069840 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.069816 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-envoy\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.070139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.070119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34009265-e234-49a1-85b7-9b95440899b2-istio-podinfo\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.074765 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.074713 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-istio-token\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.075081 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.075059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ld9b\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-kube-api-access-5ld9b\") pod \"e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.142759 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.142719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:26.282459 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.282433 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg"] Jul 07 22:24:26.284210 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:24:26.284178 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34009265_e234_49a1_85b7_9b95440899b2.slice/crio-4ac99143282683c12726bda2ebbe44d00e10d2226bda89288f56ad55be44c577 WatchSource:0}: Error finding container 4ac99143282683c12726bda2ebbe44d00e10d2226bda89288f56ad55be44c577: Status 404 returned error can't find the container with id 4ac99143282683c12726bda2ebbe44d00e10d2226bda89288f56ad55be44c577 Jul 07 22:24:26.286303 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.286269 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:26.286412 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.286374 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:26.286412 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.286404 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:26.390921 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.390876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" event={"ID":"34009265-e234-49a1-85b7-9b95440899b2","Type":"ContainerStarted","Data":"b2b1aab1852b3e47328e56b4a4240e15dca7da5a511a21b02493ff525b1dec30"} Jul 07 22:24:26.391108 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.390939 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" event={"ID":"34009265-e234-49a1-85b7-9b95440899b2","Type":"ContainerStarted","Data":"4ac99143282683c12726bda2ebbe44d00e10d2226bda89288f56ad55be44c577"} Jul 07 22:24:26.409235 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:26.409158 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" podStartSLOduration=1.409135072 podStartE2EDuration="1.409135072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:24:26.408117247 +0000 UTC m=+1545.002876405" watchObservedRunningTime="2026-07-07 22:24:26.409135072 +0000 UTC m=+1545.003894234" Jul 07 22:24:27.143112 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:27.143067 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:27.148959 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:27.148927 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:27.395824 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:27.395734 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:27.396655 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:27.396637 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:30.758433 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:30.758393 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:24:32.319679 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:32.319645 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg"] Jul 07 22:24:32.320148 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:32.319821 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" podUID="34009265-e234-49a1-85b7-9b95440899b2" containerName="istio-proxy" containerID="cri-o://b2b1aab1852b3e47328e56b4a4240e15dca7da5a511a21b02493ff525b1dec30" gracePeriod=30 Jul 07 22:24:38.325943 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.325906 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s"] Jul 07 22:24:38.329006 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.328983 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.331560 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.331434 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1d13ad38-openshift-default-dockercfg-tdrgx\"" Jul 07 22:24:38.342708 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.342661 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s"] Jul 07 22:24:38.362202 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-token\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.362351 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istiod-ca-cert\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.362351 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362276 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-credential-socket\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.362351 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-data\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.362546 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-certs\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.362546 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npxp7\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-kube-api-access-npxp7\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.362546 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-envoy\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.363045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-socket\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.363045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.362958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-podinfo\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.448621 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.448581 2562 generic.go:356] "Generic (PLEG): container finished" podID="34009265-e234-49a1-85b7-9b95440899b2" containerID="b2b1aab1852b3e47328e56b4a4240e15dca7da5a511a21b02493ff525b1dec30" exitCode=0 Jul 07 22:24:38.448788 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.448652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" event={"ID":"34009265-e234-49a1-85b7-9b95440899b2","Type":"ContainerDied","Data":"b2b1aab1852b3e47328e56b4a4240e15dca7da5a511a21b02493ff525b1dec30"} Jul 07 22:24:38.464100 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-certs\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464100 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npxp7\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-kube-api-access-npxp7\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-envoy\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-socket\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-podinfo\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-token\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istiod-ca-cert\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-credential-socket\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464373 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-data\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464769 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464544 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-certs\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464837 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-socket\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.464895 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.464856 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-data\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.465168 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.465142 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-credential-socket\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.465273 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.465219 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istiod-ca-cert\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.466713 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.466687 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-envoy\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.467255 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.467233 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-podinfo\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.472738 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.472670 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npxp7\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-kube-api-access-npxp7\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.473233 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.473200 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-token\") pod \"e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.592048 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.592017 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:38.643402 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.643357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:38.665545 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665510 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-certs\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665545 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665552 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-socket\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665589 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-envoy\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665630 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-istio-token\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665646 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34009265-e234-49a1-85b7-9b95440899b2-istiod-ca-cert\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-data\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665709 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34009265-e234-49a1-85b7-9b95440899b2-istio-podinfo\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665738 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-credential-socket\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.665767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665764 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ld9b\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-kube-api-access-5ld9b\") pod \"34009265-e234-49a1-85b7-9b95440899b2\" (UID: \"34009265-e234-49a1-85b7-9b95440899b2\") " Jul 07 22:24:38.666102 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.665854 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:38.666102 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.666055 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.666206 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.666089 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:38.666206 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.666130 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:38.666312 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.666225 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34009265-e234-49a1-85b7-9b95440899b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:24:38.666810 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.666788 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:38.668519 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.668480 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:24:38.668688 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.668665 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-kube-api-access-5ld9b" (OuterVolumeSpecName: "kube-api-access-5ld9b") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "kube-api-access-5ld9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:24:38.668750 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.668712 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:38.668815 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.668799 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34009265-e234-49a1-85b7-9b95440899b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34009265-e234-49a1-85b7-9b95440899b2" (UID: "34009265-e234-49a1-85b7-9b95440899b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:24:38.767413 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767378 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767413 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767411 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34009265-e234-49a1-85b7-9b95440899b2-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767413 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767423 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767651 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767434 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34009265-e234-49a1-85b7-9b95440899b2-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767651 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767446 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767651 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767458 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ld9b\" (UniqueName: \"kubernetes.io/projected/34009265-e234-49a1-85b7-9b95440899b2-kube-api-access-5ld9b\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767651 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767468 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.767651 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.767480 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34009265-e234-49a1-85b7-9b95440899b2-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:38.786665 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.786638 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s"] Jul 07 22:24:38.789521 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:24:38.789495 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332eb642_3d0b_4034_994d_8cce8b1dc9a9.slice/crio-e75bb867aea4c2bd31f9c28ad6667b05a8fbcd36f0afa5bb7b39912621fa6067 WatchSource:0}: Error finding container e75bb867aea4c2bd31f9c28ad6667b05a8fbcd36f0afa5bb7b39912621fa6067: Status 404 returned error can't find the container with id e75bb867aea4c2bd31f9c28ad6667b05a8fbcd36f0afa5bb7b39912621fa6067 Jul 07 22:24:38.791659 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.791625 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:38.791726 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.791701 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:38.791761 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:38.791734 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:39.455958 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.455923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" event={"ID":"332eb642-3d0b-4034-994d-8cce8b1dc9a9","Type":"ContainerStarted","Data":"2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820"} Jul 07 22:24:39.456571 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.455981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" event={"ID":"332eb642-3d0b-4034-994d-8cce8b1dc9a9","Type":"ContainerStarted","Data":"e75bb867aea4c2bd31f9c28ad6667b05a8fbcd36f0afa5bb7b39912621fa6067"} Jul 07 22:24:39.457507 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.457483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" event={"ID":"34009265-e234-49a1-85b7-9b95440899b2","Type":"ContainerDied","Data":"4ac99143282683c12726bda2ebbe44d00e10d2226bda89288f56ad55be44c577"} Jul 07 22:24:39.457634 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.457523 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg" Jul 07 22:24:39.457634 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.457534 2562 scope.go:117] "RemoveContainer" containerID="b2b1aab1852b3e47328e56b4a4240e15dca7da5a511a21b02493ff525b1dec30" Jul 07 22:24:39.485994 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.485940 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" podStartSLOduration=1.485921619 podStartE2EDuration="1.485921619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:24:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:24:39.484666475 +0000 UTC m=+1558.079425643" watchObservedRunningTime="2026-07-07 22:24:39.485921619 +0000 UTC m=+1558.080680775" Jul 07 22:24:39.499992 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.499957 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg"] Jul 07 22:24:39.506660 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.506630 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a2c4a017-openshift-default-676fd6cd65-2x8pg"] Jul 07 22:24:39.643655 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.643562 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:39.648264 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.648237 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:39.955632 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:39.955545 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34009265-e234-49a1-85b7-9b95440899b2" path="/var/lib/kubelet/pods/34009265-e234-49a1-85b7-9b95440899b2/volumes" Jul 07 22:24:40.463143 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:40.463111 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:40.464067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:40.464050 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:41.668738 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:41.668698 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:24:50.762751 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:50.762715 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s"] Jul 07 22:24:50.763222 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:50.762947 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" podUID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" containerName="istio-proxy" containerID="cri-o://2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820" gracePeriod=30 Jul 07 22:24:55.465955 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:55.465923 2562 patch_prober.go:28] interesting pod/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 07 22:24:55.466348 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:55.465982 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" podUID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 22:24:56.018960 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.018933 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:56.114341 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114297 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npxp7\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-kube-api-access-npxp7\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114376 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-podinfo\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114424 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-data\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114463 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istiod-ca-cert\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114488 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-credential-socket\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114510 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-certs\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114550 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-token\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114840 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114607 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-envoy\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114840 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114630 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-socket\") pod \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\" (UID: \"332eb642-3d0b-4034-994d-8cce8b1dc9a9\") " Jul 07 22:24:56.114840 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114780 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:56.114840 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114788 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:56.114840 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114811 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:56.115086 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114887 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:24:56.115086 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114916 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.115086 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114934 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.115086 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.114948 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.115086 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.115032 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:56.116820 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.116797 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-kube-api-access-npxp7" (OuterVolumeSpecName: "kube-api-access-npxp7") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "kube-api-access-npxp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:24:56.116820 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.116799 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:24:56.117321 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.117296 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:24:56.117321 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.117305 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "332eb642-3d0b-4034-994d-8cce8b1dc9a9" (UID: "332eb642-3d0b-4034-994d-8cce8b1dc9a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:24:56.216354 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.216292 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.216354 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.216323 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.216354 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.216357 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.216354 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.216365 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332eb642-3d0b-4034-994d-8cce8b1dc9a9-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.216622 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.216377 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npxp7\" (UniqueName: \"kubernetes.io/projected/332eb642-3d0b-4034-994d-8cce8b1dc9a9-kube-api-access-npxp7\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.216622 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.216387 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332eb642-3d0b-4034-994d-8cce8b1dc9a9-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:24:56.536552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.536458 2562 generic.go:356] "Generic (PLEG): container finished" podID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" containerID="2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820" exitCode=0 Jul 07 22:24:56.536552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.536507 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" Jul 07 22:24:56.536552 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.536518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" event={"ID":"332eb642-3d0b-4034-994d-8cce8b1dc9a9","Type":"ContainerDied","Data":"2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820"} Jul 07 22:24:56.537138 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.536562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s" event={"ID":"332eb642-3d0b-4034-994d-8cce8b1dc9a9","Type":"ContainerDied","Data":"e75bb867aea4c2bd31f9c28ad6667b05a8fbcd36f0afa5bb7b39912621fa6067"} Jul 07 22:24:56.537138 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.536583 2562 scope.go:117] "RemoveContainer" containerID="2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820" Jul 07 22:24:56.545418 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.545397 2562 scope.go:117] "RemoveContainer" containerID="2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820" Jul 07 22:24:56.545725 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:24:56.545707 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820\": container with ID starting with 2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820 not found: ID does not exist" containerID="2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820" Jul 07 22:24:56.545779 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.545735 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820"} err="failed to get container status \"2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820\": rpc error: code = NotFound desc = could not find container \"2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820\": container with ID starting with 2a576da0deae4d035b8989a03e548a4291f6e6a1c216e635bbce0cc541483820 not found: ID does not exist" Jul 07 22:24:56.556238 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.556206 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s"] Jul 07 22:24:56.560155 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.560133 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1d13ad38-openshift-default-7f468bcb6f-7zj2s"] Jul 07 22:24:56.660135 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660091 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4"] Jul 07 22:24:56.660447 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660431 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34009265-e234-49a1-85b7-9b95440899b2" containerName="istio-proxy" Jul 07 22:24:56.660447 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660447 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34009265-e234-49a1-85b7-9b95440899b2" containerName="istio-proxy" Jul 07 22:24:56.660598 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660458 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" containerName="istio-proxy" Jul 07 22:24:56.660598 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660464 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" containerName="istio-proxy" Jul 07 22:24:56.660598 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660552 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" containerName="istio-proxy" Jul 07 22:24:56.660598 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.660563 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34009265-e234-49a1-85b7-9b95440899b2" containerName="istio-proxy" Jul 07 22:24:56.663051 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.663028 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.665538 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.665512 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e6aec1d1-openshift-default-dockercfg-fgc77\"" Jul 07 22:24:56.675740 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.675711 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4"] Jul 07 22:24:56.720669 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-data\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.720669 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-socket\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.720888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-envoy\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.720888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-credential-socket\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.720888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf60ed4-cc3f-4191-aa47-39089589c6da-istiod-ca-cert\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.720888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4v74\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-kube-api-access-p4v74\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.720888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-certs\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.721035 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-token\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.721035 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.720968 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-podinfo\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821822 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-certs\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821969 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-token\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821969 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-podinfo\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821969 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-data\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821969 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-socket\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821969 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-envoy\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.821969 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-credential-socket\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.822229 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.821980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf60ed4-cc3f-4191-aa47-39089589c6da-istiod-ca-cert\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.822229 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.822023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4v74\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-kube-api-access-p4v74\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.822359 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.822266 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-certs\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.822488 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.822457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-socket\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.822590 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.822537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-credential-socket\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.822654 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.822587 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-data\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.823082 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.823059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf60ed4-cc3f-4191-aa47-39089589c6da-istiod-ca-cert\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.824554 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.824522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-envoy\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.824716 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.824699 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-podinfo\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.829595 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.829569 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-token\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.829794 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.829774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4v74\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-kube-api-access-p4v74\") pod \"e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:56.974917 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:56.974880 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:57.113415 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.113385 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4"] Jul 07 22:24:57.116064 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:24:57.116034 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf60ed4_cc3f_4191_aa47_39089589c6da.slice/crio-28593f3afbfd81f2216588a254a1b5dcbc1ac42746c195ec805a8a4f5aee079d WatchSource:0}: Error finding container 28593f3afbfd81f2216588a254a1b5dcbc1ac42746c195ec805a8a4f5aee079d: Status 404 returned error can't find the container with id 28593f3afbfd81f2216588a254a1b5dcbc1ac42746c195ec805a8a4f5aee079d Jul 07 22:24:57.118188 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.118149 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:57.118276 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.118240 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:57.118346 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.118314 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:24:57.544974 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.544884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" event={"ID":"bcf60ed4-cc3f-4191-aa47-39089589c6da","Type":"ContainerStarted","Data":"d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1"} Jul 07 22:24:57.544974 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.544933 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" event={"ID":"bcf60ed4-cc3f-4191-aa47-39089589c6da","Type":"ContainerStarted","Data":"28593f3afbfd81f2216588a254a1b5dcbc1ac42746c195ec805a8a4f5aee079d"} Jul 07 22:24:57.585665 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.585608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" podStartSLOduration=1.585586876 podStartE2EDuration="1.585586876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:24:57.584849923 +0000 UTC m=+1576.179609081" watchObservedRunningTime="2026-07-07 22:24:57.585586876 +0000 UTC m=+1576.180346035" Jul 07 22:24:57.954193 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.954154 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="332eb642-3d0b-4034-994d-8cce8b1dc9a9" path="/var/lib/kubelet/pods/332eb642-3d0b-4034-994d-8cce8b1dc9a9/volumes" Jul 07 22:24:57.975456 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:57.975419 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:58.980506 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:58.980472 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:59.554799 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:59.554763 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:24:59.555759 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:24:59.555735 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:25:00.569352 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:00.569313 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:25:02.413735 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:02.413699 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4"] Jul 07 22:25:02.571365 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:02.570981 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" podUID="bcf60ed4-cc3f-4191-aa47-39089589c6da" containerName="istio-proxy" containerID="cri-o://d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1" gracePeriod=30 Jul 07 22:25:08.523741 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.523709 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:25:08.604823 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.604791 2562 generic.go:356] "Generic (PLEG): container finished" podID="bcf60ed4-cc3f-4191-aa47-39089589c6da" containerID="d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1" exitCode=0 Jul 07 22:25:08.605036 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.604845 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" Jul 07 22:25:08.605036 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.604880 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" event={"ID":"bcf60ed4-cc3f-4191-aa47-39089589c6da","Type":"ContainerDied","Data":"d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1"} Jul 07 22:25:08.605036 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.604936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4" event={"ID":"bcf60ed4-cc3f-4191-aa47-39089589c6da","Type":"ContainerDied","Data":"28593f3afbfd81f2216588a254a1b5dcbc1ac42746c195ec805a8a4f5aee079d"} Jul 07 22:25:08.605036 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.604960 2562 scope.go:117] "RemoveContainer" containerID="d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1" Jul 07 22:25:08.616288 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.616264 2562 scope.go:117] "RemoveContainer" containerID="d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1" Jul 07 22:25:08.616645 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:25:08.616618 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1\": container with ID starting with d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1 not found: ID does not exist" containerID="d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1" Jul 07 22:25:08.616726 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.616656 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1"} err="failed to get container status \"d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1\": rpc error: code = NotFound desc = could not find container \"d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1\": container with ID starting with d41b07f66929662eea0ce044ce85ab480a590174d88348f317ae81d6498e66d1 not found: ID does not exist" Jul 07 22:25:08.633416 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633378 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-socket\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633605 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633439 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-token\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633605 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633474 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-data\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633605 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633499 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-certs\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633605 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633526 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-podinfo\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633605 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633599 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf60ed4-cc3f-4191-aa47-39089589c6da-istiod-ca-cert\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633856 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633652 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-credential-socket\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633856 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633671 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:25:08.633856 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633684 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-envoy\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633856 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633793 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4v74\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-kube-api-access-p4v74\") pod \"bcf60ed4-cc3f-4191-aa47-39089589c6da\" (UID: \"bcf60ed4-cc3f-4191-aa47-39089589c6da\") " Jul 07 22:25:08.633856 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633815 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:25:08.634107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.633995 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:25:08.634107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.634061 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.634107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.634081 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.634107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.634095 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.634107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.634093 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf60ed4-cc3f-4191-aa47-39089589c6da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:25:08.634568 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.634543 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-data" (OuterVolumeSpecName: "istio-data") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:25:08.635930 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.635903 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:25:08.636043 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.635941 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:25:08.636043 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.636022 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-kube-api-access-p4v74" (OuterVolumeSpecName: "kube-api-access-p4v74") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "kube-api-access-p4v74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:25:08.636114 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.636056 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bcf60ed4-cc3f-4191-aa47-39089589c6da" (UID: "bcf60ed4-cc3f-4191-aa47-39089589c6da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:25:08.735394 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.735354 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcf60ed4-cc3f-4191-aa47-39089589c6da-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.735394 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.735391 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.735394 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.735400 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4v74\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-kube-api-access-p4v74\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.735666 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.735411 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.735666 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.735420 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.735666 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.735427 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcf60ed4-cc3f-4191-aa47-39089589c6da-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:25:08.931898 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.931862 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4"] Jul 07 22:25:08.944699 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:08.944650 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e6aec1d1-openshift-default-7d8d97d687-gw7h4"] Jul 07 22:25:09.954630 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:09.954594 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf60ed4-cc3f-4191-aa47-39089589c6da" path="/var/lib/kubelet/pods/bcf60ed4-cc3f-4191-aa47-39089589c6da/volumes" Jul 07 22:25:10.065320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:25:10.065285 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:26:31.582770 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:31.582731 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:26:40.966360 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:40.966306 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:26:43.800961 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.800926 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg"] Jul 07 22:26:43.801458 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.801399 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf60ed4-cc3f-4191-aa47-39089589c6da" containerName="istio-proxy" Jul 07 22:26:43.801458 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.801419 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf60ed4-cc3f-4191-aa47-39089589c6da" containerName="istio-proxy" Jul 07 22:26:43.801568 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.801510 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf60ed4-cc3f-4191-aa47-39089589c6da" containerName="istio-proxy" Jul 07 22:26:43.803387 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.803365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.805978 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.805946 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3d19c1ee-openshift-default-dockercfg-p5wzz\"" Jul 07 22:26:43.823654 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.823620 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg"] Jul 07 22:26:43.842080 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-certs\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842273 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istiod-ca-cert\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842273 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbsj8\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-kube-api-access-cbsj8\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842273 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-data\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842273 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842269 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-envoy\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842452 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-podinfo\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842452 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-credential-socket\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842527 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-socket\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.842527 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.842479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-token\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.943810 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943768 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-certs\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.943810 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istiod-ca-cert\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943829 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbsj8\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-kube-api-access-cbsj8\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-data\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-envoy\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-podinfo\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.943962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-credential-socket\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-socket\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-token\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944487 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944436 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-data\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944487 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944472 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-credential-socket\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944675 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-socket\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944675 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944564 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-certs\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.944787 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.944764 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istiod-ca-cert\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.946715 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.946692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-podinfo\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.946715 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.946710 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-envoy\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.951708 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.951678 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-token\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:43.952187 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:43.952165 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbsj8\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-kube-api-access-cbsj8\") pod \"e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:44.115239 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:44.115191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:44.265816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:44.265782 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg"] Jul 07 22:26:44.269240 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:26:44.269209 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b1ccfd_1ee1_49c2_9fdb_1f4da4702143.slice/crio-c0d0914aad7daca6fbcc64e07ab8b2814831d5fee967bfcdedd262755d9e67e6 WatchSource:0}: Error finding container c0d0914aad7daca6fbcc64e07ab8b2814831d5fee967bfcdedd262755d9e67e6: Status 404 returned error can't find the container with id c0d0914aad7daca6fbcc64e07ab8b2814831d5fee967bfcdedd262755d9e67e6 Jul 07 22:26:44.271465 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:44.271415 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:26:44.271562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:44.271515 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:26:44.271562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:44.271555 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:26:45.020266 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:45.020230 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" event={"ID":"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143","Type":"ContainerStarted","Data":"f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21"} Jul 07 22:26:45.020706 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:45.020284 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" event={"ID":"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143","Type":"ContainerStarted","Data":"c0d0914aad7daca6fbcc64e07ab8b2814831d5fee967bfcdedd262755d9e67e6"} Jul 07 22:26:45.042022 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:45.041969 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" podStartSLOduration=2.041950437 podStartE2EDuration="2.041950437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:26:45.041165766 +0000 UTC m=+1683.635924924" watchObservedRunningTime="2026-07-07 22:26:45.041950437 +0000 UTC m=+1683.636709593" Jul 07 22:26:45.115713 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:45.115667 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:45.120669 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:45.120643 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:46.024964 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:46.024933 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:46.025932 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:46.025914 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:26:51.659490 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:26:51.659443 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:27:00.088779 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:00.088736 2562 generic.go:356] "Generic (PLEG): container finished" podID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerID="f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21" exitCode=0 Jul 07 22:27:00.089287 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:00.088796 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" event={"ID":"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143","Type":"ContainerDied","Data":"f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21"} Jul 07 22:27:00.089287 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:00.089159 2562 scope.go:117] "RemoveContainer" containerID="f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21" Jul 07 22:27:00.090226 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:00.090201 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:00.090305 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:00.090267 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:00.090305 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:00.090294 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:01.025116 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:01.025076 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:01.095029 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:01.094990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" event={"ID":"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143","Type":"ContainerStarted","Data":"7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527"} Jul 07 22:27:01.115410 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:01.115370 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:01.116599 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:01.116576 2562 patch_prober.go:28] interesting pod/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 22:27:01.116698 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:01.116625 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 22:27:02.116482 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:02.116444 2562 patch_prober.go:28] interesting pod/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 22:27:02.116906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:02.116501 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 22:27:02.464993 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:02.464880 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:27:03.115748 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:03.115703 2562 patch_prober.go:28] interesting pod/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 22:27:03.116008 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:03.115776 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 22:27:04.115676 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:04.115635 2562 patch_prober.go:28] interesting pod/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 22:27:04.116158 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:04.115699 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 22:27:05.120558 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:05.120529 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:05.120987 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:05.120863 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:05.121622 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:05.121607 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:05.305749 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:05.305718 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg"] Jul 07 22:27:06.116347 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:06.116308 2562 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" secret="" err="secret \"e2e-mt-3d19c1ee-openshift-default-dockercfg-p5wzz\" not found" Jul 07 22:27:07.120670 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:07.120623 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" containerID="cri-o://7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527" gracePeriod=30 Jul 07 22:27:12.375066 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.375040 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:12.503665 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503560 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbsj8\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-kube-api-access-cbsj8\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.503665 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503613 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-certs\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.503665 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503659 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-envoy\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503771 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-socket\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503809 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-token\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503892 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-credential-socket\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503919 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istiod-ca-cert\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503975 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-data\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.503963 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:27:12.504010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504005 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-podinfo\") pod \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\" (UID: \"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143\") " Jul 07 22:27:12.504562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504021 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:27:12.504562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504200 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:27:12.504562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504270 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.504562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504276 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:27:12.504562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504282 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.504562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.504323 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.505243 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.505213 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:27:12.506107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.506075 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-kube-api-access-cbsj8" (OuterVolumeSpecName: "kube-api-access-cbsj8") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "kube-api-access-cbsj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:27:12.506215 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.506146 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:27:12.506274 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.506245 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:27:12.506397 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.506381 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" (UID: "b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:27:12.604920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.604882 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.604920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.604911 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.604920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.604920 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.604920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.604929 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbsj8\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-kube-api-access-cbsj8\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.605183 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.604940 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:12.605183 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:12.604948 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:27:13.148653 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.148612 2562 generic.go:356] "Generic (PLEG): container finished" podID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerID="7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527" exitCode=0 Jul 07 22:27:13.148851 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.148667 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" Jul 07 22:27:13.148851 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.148696 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" event={"ID":"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143","Type":"ContainerDied","Data":"7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527"} Jul 07 22:27:13.148851 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.148752 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg" event={"ID":"b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143","Type":"ContainerDied","Data":"c0d0914aad7daca6fbcc64e07ab8b2814831d5fee967bfcdedd262755d9e67e6"} Jul 07 22:27:13.148851 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.148775 2562 scope.go:117] "RemoveContainer" containerID="7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527" Jul 07 22:27:13.158217 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.158199 2562 scope.go:117] "RemoveContainer" containerID="f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21" Jul 07 22:27:13.166392 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.166372 2562 scope.go:117] "RemoveContainer" containerID="7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527" Jul 07 22:27:13.166651 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:27:13.166633 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527\": container with ID starting with 7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527 not found: ID does not exist" containerID="7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527" Jul 07 22:27:13.166697 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.166661 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527"} err="failed to get container status \"7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527\": rpc error: code = NotFound desc = could not find container \"7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527\": container with ID starting with 7088a7194d58b5b69e9902b5bf7a54304b4d07f271dd41f17349182a860b5527 not found: ID does not exist" Jul 07 22:27:13.166697 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.166680 2562 scope.go:117] "RemoveContainer" containerID="f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21" Jul 07 22:27:13.166894 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:27:13.166879 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21\": container with ID starting with f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21 not found: ID does not exist" containerID="f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21" Jul 07 22:27:13.166934 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.166899 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21"} err="failed to get container status \"f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21\": rpc error: code = NotFound desc = could not find container \"f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21\": container with ID starting with f6c06e6c5d5f435baabf23ad8a93ce20f9ab4087762891b6a44e09ae0b03ae21 not found: ID does not exist" Jul 07 22:27:13.169941 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.169917 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg"] Jul 07 22:27:13.173152 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.173131 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3d19c1ee-openshift-default-8984b8967-h6cqg"] Jul 07 22:27:13.954511 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:13.954472 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" path="/var/lib/kubelet/pods/b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143/volumes" Jul 07 22:27:24.885613 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.885570 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl"] Jul 07 22:27:24.886076 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.886061 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" Jul 07 22:27:24.886129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.886080 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" Jul 07 22:27:24.886204 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.886193 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" Jul 07 22:27:24.886242 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.886207 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" Jul 07 22:27:24.889157 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.889133 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.891706 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.891683 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5ebf133b-openshift-default-dockercfg-6j2wk\"" Jul 07 22:27:24.899627 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.899595 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl"] Jul 07 22:27:24.903123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56r56\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-kube-api-access-56r56\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-podinfo\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-certs\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903241 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-data\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-token\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-envoy\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istiod-ca-cert\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-socket\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:24.903518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:24.903446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-credential-socket\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004319 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-certs\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-data\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-token\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-envoy\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istiod-ca-cert\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004518 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-socket\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004823 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-credential-socket\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004823 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56r56\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-kube-api-access-56r56\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004823 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-podinfo\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004993 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-certs\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.004993 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.004913 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-credential-socket\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.005108 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.005079 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-socket\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.005282 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.005263 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-data\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.005521 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.005496 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istiod-ca-cert\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.006977 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.006955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-envoy\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.007271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.007254 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-podinfo\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.014129 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.014094 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56r56\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-kube-api-access-56r56\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.014301 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.014138 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-token\") pod \"e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.200461 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.200372 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:25.351572 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.351544 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl"] Jul 07 22:27:25.354215 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:27:25.354184 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f0a1e5_f4a1_416e_a3af_0eb1ad90d665.slice/crio-efe22255d3dc09d45fee56410cbd8f166c9e112fa3d3c93ff351dc4280b0a552 WatchSource:0}: Error finding container efe22255d3dc09d45fee56410cbd8f166c9e112fa3d3c93ff351dc4280b0a552: Status 404 returned error can't find the container with id efe22255d3dc09d45fee56410cbd8f166c9e112fa3d3c93ff351dc4280b0a552 Jul 07 22:27:25.356516 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.356488 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:25.356625 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.356553 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:25.356625 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:25.356583 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:26.206599 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:26.206562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" event={"ID":"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665","Type":"ContainerStarted","Data":"9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a"} Jul 07 22:27:26.207038 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:26.206620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" event={"ID":"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665","Type":"ContainerStarted","Data":"efe22255d3dc09d45fee56410cbd8f166c9e112fa3d3c93ff351dc4280b0a552"} Jul 07 22:27:26.230203 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:26.230150 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podStartSLOduration=2.230132883 podStartE2EDuration="2.230132883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:27:26.228815764 +0000 UTC m=+1724.823574923" watchObservedRunningTime="2026-07-07 22:27:26.230132883 +0000 UTC m=+1724.824892040" Jul 07 22:27:27.201289 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:27.201242 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:27.206650 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:27.206619 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:27.210926 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:27.210887 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:27.211956 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:27.211936 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:33.762084 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:33.762040 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:27:42.212606 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.212570 2562 patch_prober.go:28] interesting pod/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 22:27:42.213018 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.212630 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 22:27:42.281261 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.281162 2562 generic.go:356] "Generic (PLEG): container finished" podID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerID="9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a" exitCode=0 Jul 07 22:27:42.281261 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.281222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" event={"ID":"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665","Type":"ContainerDied","Data":"9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a"} Jul 07 22:27:42.281700 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.281682 2562 scope.go:117] "RemoveContainer" containerID="9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a" Jul 07 22:27:42.282822 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.282787 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:42.282919 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.282879 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:42.282959 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:42.282925 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:27:43.290792 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:43.290751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" event={"ID":"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665","Type":"ContainerStarted","Data":"a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a"} Jul 07 22:27:44.201446 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:44.201396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:44.201724 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:44.201699 2562 patch_prober.go:28] interesting pod/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 22:27:44.201823 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:44.201761 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 22:27:45.200882 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:45.200841 2562 patch_prober.go:28] interesting pod/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 22:27:45.201294 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:45.200903 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 22:27:45.862259 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:45.862223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:27:46.201181 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:46.201088 2562 patch_prober.go:28] interesting pod/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 22:27:46.201181 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:46.201156 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 22:27:47.201599 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:47.201554 2562 patch_prober.go:28] interesting pod/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 22:27:47.202007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:47.201617 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 22:27:48.204960 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:48.204923 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:48.205383 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:48.205306 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:48.206034 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:48.206018 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:27:52.967938 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:27:52.967897 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:28:06.559262 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:06.559215 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:28:45.364977 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:45.364942 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:28:48.902651 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:48.902614 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl"] Jul 07 22:28:48.903067 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:48.902837 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" containerID="cri-o://a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a" gracePeriod=30 Jul 07 22:28:54.148978 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.148949 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:28:54.192711 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.192681 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-envoy\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.192885 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.192723 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-credential-socket\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.192885 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.192755 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56r56\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-kube-api-access-56r56\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.192885 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.192772 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-certs\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.192885 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.192796 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-socket\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.193136 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.193105 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:28:54.193197 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.193124 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:28:54.193197 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.193113 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:28:54.194997 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.194977 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-kube-api-access-56r56" (OuterVolumeSpecName: "kube-api-access-56r56") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "kube-api-access-56r56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:28:54.195063 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.195002 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:28:54.293223 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293176 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-token\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.293223 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293222 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-podinfo\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.293223 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293242 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-data\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.293564 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293265 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istiod-ca-cert\") pod \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\" (UID: \"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665\") " Jul 07 22:28:54.293564 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293474 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56r56\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-kube-api-access-56r56\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.293564 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293492 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.293564 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293508 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.293564 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293521 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.293564 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293533 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.293808 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293654 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:28:54.293933 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.293911 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-data" (OuterVolumeSpecName: "istio-data") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:28:54.295547 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.295523 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-token" (OuterVolumeSpecName: "istio-token") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:28:54.295629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.295547 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" (UID: "98f0a1e5-f4a1-416e-a3af-0eb1ad90d665"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:28:54.394010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.393921 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.394010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.393951 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.394010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.393961 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.394010 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.393968 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:28:54.602559 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.602524 2562 generic.go:356] "Generic (PLEG): container finished" podID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerID="a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a" exitCode=0 Jul 07 22:28:54.602734 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.602567 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" Jul 07 22:28:54.602734 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.602566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" event={"ID":"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665","Type":"ContainerDied","Data":"a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a"} Jul 07 22:28:54.602734 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.602690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl" event={"ID":"98f0a1e5-f4a1-416e-a3af-0eb1ad90d665","Type":"ContainerDied","Data":"efe22255d3dc09d45fee56410cbd8f166c9e112fa3d3c93ff351dc4280b0a552"} Jul 07 22:28:54.602734 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.602715 2562 scope.go:117] "RemoveContainer" containerID="a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a" Jul 07 22:28:54.612186 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.612168 2562 scope.go:117] "RemoveContainer" containerID="9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a" Jul 07 22:28:54.620717 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.620697 2562 scope.go:117] "RemoveContainer" containerID="a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a" Jul 07 22:28:54.621001 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:28:54.620977 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a\": container with ID starting with a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a not found: ID does not exist" containerID="a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a" Jul 07 22:28:54.621050 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.621011 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a"} err="failed to get container status \"a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a\": rpc error: code = NotFound desc = could not find container \"a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a\": container with ID starting with a58d9fce79d2aa2784916d0cdc7b1564f6b169f5e3b4b0a00b5b0f49fc1d4a1a not found: ID does not exist" Jul 07 22:28:54.621050 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.621028 2562 scope.go:117] "RemoveContainer" containerID="9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a" Jul 07 22:28:54.621307 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:28:54.621289 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a\": container with ID starting with 9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a not found: ID does not exist" containerID="9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a" Jul 07 22:28:54.621439 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.621313 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a"} err="failed to get container status \"9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a\": rpc error: code = NotFound desc = could not find container \"9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a\": container with ID starting with 9d1d3c3b9de1e30a4bfab0c73d970156d811463754c3edbdd56b5e576ab5329a not found: ID does not exist" Jul 07 22:28:54.628215 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.628186 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl"] Jul 07 22:28:54.631583 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:54.631556 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5ebf133b-openshift-default-68d4988b59-fxlnl"] Jul 07 22:28:55.954731 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:28:55.954688 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" path="/var/lib/kubelet/pods/98f0a1e5-f4a1-416e-a3af-0eb1ad90d665/volumes" Jul 07 22:29:43.243644 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.243609 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb"] Jul 07 22:29:43.244065 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244048 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" Jul 07 22:29:43.244111 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244068 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b1ccfd-1ee1-49c2-9fdb-1f4da4702143" containerName="istio-proxy" Jul 07 22:29:43.244111 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244087 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" Jul 07 22:29:43.244111 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244095 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" Jul 07 22:29:43.244111 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244104 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" Jul 07 22:29:43.244231 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244113 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" Jul 07 22:29:43.244231 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244192 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" Jul 07 22:29:43.244231 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.244202 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f0a1e5-f4a1-416e-a3af-0eb1ad90d665" containerName="istio-proxy" Jul 07 22:29:43.248635 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.248612 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.251059 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.251035 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cd9893-openshift-default-dockercfg-sr7tg\"" Jul 07 22:29:43.257582 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.257553 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb"] Jul 07 22:29:43.308369 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-certs\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308561 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7mqq\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-kube-api-access-f7mqq\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308561 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308445 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-envoy\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308561 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12e58bb2-eb12-43e3-86ab-6feefe18a325-istiod-ca-cert\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308561 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-credential-socket\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308561 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308549 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-socket\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308775 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-token\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308775 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-podinfo\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.308775 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.308672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-data\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.409824 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.409780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-socket\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410026 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.409840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-token\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410026 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.409879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-podinfo\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410026 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.409911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-data\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410026 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.409942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-certs\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410241 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410146 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7mqq\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-kube-api-access-f7mqq\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410241 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-envoy\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410381 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410251 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-socket\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410381 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12e58bb2-eb12-43e3-86ab-6feefe18a325-istiod-ca-cert\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410381 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410283 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-certs\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410381 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410288 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-credential-socket\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410381 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410318 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-data\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410600 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410584 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-credential-socket\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.410886 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.410870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12e58bb2-eb12-43e3-86ab-6feefe18a325-istiod-ca-cert\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.412688 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.412660 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-envoy\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.412998 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.412979 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-podinfo\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.418433 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.418403 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7mqq\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-kube-api-access-f7mqq\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.418612 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.418593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-token\") pod \"e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.560644 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.560550 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:43.696302 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.696275 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb"] Jul 07 22:29:43.698886 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:29:43.698862 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e58bb2_eb12_43e3_86ab_6feefe18a325.slice/crio-38ffa79a33282b354d1bd84a49362bf9166b506d374908d9bfd03249c61ad65b WatchSource:0}: Error finding container 38ffa79a33282b354d1bd84a49362bf9166b506d374908d9bfd03249c61ad65b: Status 404 returned error can't find the container with id 38ffa79a33282b354d1bd84a49362bf9166b506d374908d9bfd03249c61ad65b Jul 07 22:29:43.700951 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.700922 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:29:43.701041 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.700984 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:29:43.701041 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.701012 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:29:43.812625 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.812524 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" event={"ID":"12e58bb2-eb12-43e3-86ab-6feefe18a325","Type":"ContainerStarted","Data":"b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc"} Jul 07 22:29:43.812625 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.812586 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" event={"ID":"12e58bb2-eb12-43e3-86ab-6feefe18a325","Type":"ContainerStarted","Data":"38ffa79a33282b354d1bd84a49362bf9166b506d374908d9bfd03249c61ad65b"} Jul 07 22:29:43.832051 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:43.831995 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podStartSLOduration=0.831978106 podStartE2EDuration="831.978106ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:29:43.830638861 +0000 UTC m=+1862.425398019" watchObservedRunningTime="2026-07-07 22:29:43.831978106 +0000 UTC m=+1862.426737241" Jul 07 22:29:44.561107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:44.561063 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:44.565933 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:44.565904 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:44.816923 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:44.816828 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:44.817757 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:44.817738 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:29:50.862635 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.862595 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp"] Jul 07 22:29:50.866173 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.866145 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.868477 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.868454 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-fe6b32-openshift-default-dockercfg-gmlvr\"" Jul 07 22:29:50.877987 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.877961 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp"] Jul 07 22:29:50.973183 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-envoy\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973413 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973276 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-data\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973413 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-certs\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973535 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-socket\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973591 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-credential-socket\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973591 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istiod-ca-cert\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973681 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xczsn\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-kube-api-access-xczsn\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973681 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-podinfo\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:50.973783 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:50.973692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-token\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074624 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-envoy\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074624 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-data\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-certs\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-socket\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-credential-socket\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istiod-ca-cert\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xczsn\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-kube-api-access-xczsn\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-podinfo\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.074888 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.074830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-token\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.075235 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.075218 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-data\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.075468 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.075442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-credential-socket\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.075649 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.075625 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-certs\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.075755 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.075675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-socket\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.075871 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.075845 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istiod-ca-cert\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.077468 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.077437 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-envoy\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.078016 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.077995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-podinfo\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.083309 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.083272 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-token\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.083490 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.083463 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xczsn\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-kube-api-access-xczsn\") pod \"e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.179668 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.179571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:51.321164 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.321140 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp"] Jul 07 22:29:51.323408 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:29:51.323380 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a2cb3c_dc9d_4bb9_b7cb_d2a9d4d04a33.slice/crio-4fe6407865da10fca6849c830558b1498fffcb7c36eda74c9eac633a23f36a2f WatchSource:0}: Error finding container 4fe6407865da10fca6849c830558b1498fffcb7c36eda74c9eac633a23f36a2f: Status 404 returned error can't find the container with id 4fe6407865da10fca6849c830558b1498fffcb7c36eda74c9eac633a23f36a2f Jul 07 22:29:51.325465 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.325437 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:29:51.325575 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.325512 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:29:51.325575 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.325555 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:29:51.848087 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.848048 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" event={"ID":"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33","Type":"ContainerStarted","Data":"9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe"} Jul 07 22:29:51.848283 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.848103 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" event={"ID":"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33","Type":"ContainerStarted","Data":"4fe6407865da10fca6849c830558b1498fffcb7c36eda74c9eac633a23f36a2f"} Jul 07 22:29:51.867225 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:51.867170 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podStartSLOduration=1.867150283 podStartE2EDuration="1.867150283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:29:51.865471286 +0000 UTC m=+1870.460230448" watchObservedRunningTime="2026-07-07 22:29:51.867150283 +0000 UTC m=+1870.461909439" Jul 07 22:29:52.179823 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:52.179729 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:52.184863 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:52.184831 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:52.852701 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:52.852670 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:52.853743 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:52.853722 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:29:53.659780 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:29:53.659743 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:30:00.858551 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:00.858504 2562 patch_prober.go:28] interesting pod/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 22:30:00.858963 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:00.858564 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 22:30:01.897214 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:01.897177 2562 generic.go:356] "Generic (PLEG): container finished" podID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerID="b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc" exitCode=0 Jul 07 22:30:01.897693 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:01.897251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" event={"ID":"12e58bb2-eb12-43e3-86ab-6feefe18a325","Type":"ContainerDied","Data":"b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc"} Jul 07 22:30:01.897693 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:01.897662 2562 scope.go:117] "RemoveContainer" containerID="b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc" Jul 07 22:30:01.898776 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:01.898750 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:30:01.898836 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:01.898817 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:30:01.898871 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:01.898849 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:30:02.903222 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:02.903141 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" event={"ID":"12e58bb2-eb12-43e3-86ab-6feefe18a325","Type":"ContainerStarted","Data":"5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48"} Jul 07 22:30:03.561153 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:03.561116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:30:03.561408 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:03.561383 2562 patch_prober.go:28] interesting pod/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 22:30:03.561499 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:03.561460 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 22:30:04.561344 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:04.561303 2562 patch_prober.go:28] interesting pod/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 22:30:04.561765 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:04.561392 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 22:30:04.764752 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:04.764720 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:30:05.561009 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:05.560968 2562 patch_prober.go:28] interesting pod/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 22:30:05.561185 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:05.561050 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 22:30:06.561275 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:06.561241 2562 patch_prober.go:28] interesting pod/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 22:30:06.561702 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:06.561296 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 22:30:07.565110 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:07.565080 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:30:07.565537 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:07.565378 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:30:07.566174 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:07.566153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:30:08.496320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.495472 2562 patch_prober.go:28] interesting pod/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": read tcp 10.133.0.2:39656->10.133.0.31:15021: read: connection reset by peer" start-of-body= Jul 07 22:30:08.496320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.495529 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": read tcp 10.133.0.2:39656->10.133.0.31:15021: read: connection reset by peer" Jul 07 22:30:08.936312 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.936278 2562 generic.go:356] "Generic (PLEG): container finished" podID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerID="9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe" exitCode=0 Jul 07 22:30:08.936732 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.936356 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" event={"ID":"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33","Type":"ContainerDied","Data":"9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe"} Jul 07 22:30:08.936890 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.936871 2562 scope.go:117] "RemoveContainer" containerID="9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe" Jul 07 22:30:08.937896 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.937870 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:30:08.937955 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.937942 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:30:08.937991 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:08.937980 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:30:09.943523 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:09.943483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" event={"ID":"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33","Type":"ContainerStarted","Data":"8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3"} Jul 07 22:30:10.179906 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:10.179854 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:30:10.180315 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:10.180286 2562 patch_prober.go:28] interesting pod/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 22:30:10.180430 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:10.180379 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 22:30:11.180597 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:11.180545 2562 patch_prober.go:28] interesting pod/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 22:30:11.181138 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:11.180623 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 22:30:12.180433 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:12.180384 2562 patch_prober.go:28] interesting pod/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 22:30:12.180662 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:12.180470 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 22:30:12.967076 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:12.967021 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:30:13.180681 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:13.180632 2562 patch_prober.go:28] interesting pod/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 22:30:13.181300 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:13.180705 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 22:30:14.184468 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:14.184437 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:30:14.184897 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:14.184802 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:30:14.185710 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:14.185691 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:30:24.763873 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:24.763837 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:30:35.770742 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:35.770697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:30:57.466107 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:30:57.466066 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:31:35.457848 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:35.457745 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb"] Jul 07 22:31:35.458471 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:35.458057 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" containerID="cri-o://5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48" gracePeriod=30 Jul 07 22:31:37.568259 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:37.568220 2562 patch_prober.go:28] interesting pod/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 22:31:37.568673 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:37.568284 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 22:31:40.713060 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.713021 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:31:40.802985 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.802881 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-token\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.802985 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.802942 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12e58bb2-eb12-43e3-86ab-6feefe18a325-istiod-ca-cert\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.802985 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.802983 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-socket\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803018 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-certs\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803045 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7mqq\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-kube-api-access-f7mqq\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803135 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-podinfo\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803169 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-data\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-envoy\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803271 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803257 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-credential-socket\") pod \"12e58bb2-eb12-43e3-86ab-6feefe18a325\" (UID: \"12e58bb2-eb12-43e3-86ab-6feefe18a325\") " Jul 07 22:31:40.803612 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803381 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:40.803612 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803398 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:40.803725 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803614 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:40.803725 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803706 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.803801 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803758 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.804017 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.803991 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e58bb2-eb12-43e3-86ab-6feefe18a325-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:31:40.804414 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.804380 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-data" (OuterVolumeSpecName: "istio-data") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:40.805547 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.805520 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:31:40.805638 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.805607 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-token" (OuterVolumeSpecName: "istio-token") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:31:40.806033 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.806010 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-kube-api-access-f7mqq" (OuterVolumeSpecName: "kube-api-access-f7mqq") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "kube-api-access-f7mqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:31:40.806033 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.806021 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12e58bb2-eb12-43e3-86ab-6feefe18a325" (UID: "12e58bb2-eb12-43e3-86ab-6feefe18a325"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:40.904795 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904750 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7mqq\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-kube-api-access-f7mqq\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.904795 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904781 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.904795 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904790 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.904795 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904799 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.904795 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904807 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12e58bb2-eb12-43e3-86ab-6feefe18a325-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.905098 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904815 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12e58bb2-eb12-43e3-86ab-6feefe18a325-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:40.905098 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:40.904823 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12e58bb2-eb12-43e3-86ab-6feefe18a325-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:41.345298 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.345267 2562 generic.go:356] "Generic (PLEG): container finished" podID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerID="5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48" exitCode=0 Jul 07 22:31:41.345538 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.345345 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" Jul 07 22:31:41.345538 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.345361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" event={"ID":"12e58bb2-eb12-43e3-86ab-6feefe18a325","Type":"ContainerDied","Data":"5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48"} Jul 07 22:31:41.345538 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.345420 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb" event={"ID":"12e58bb2-eb12-43e3-86ab-6feefe18a325","Type":"ContainerDied","Data":"38ffa79a33282b354d1bd84a49362bf9166b506d374908d9bfd03249c61ad65b"} Jul 07 22:31:41.345538 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.345443 2562 scope.go:117] "RemoveContainer" containerID="5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48" Jul 07 22:31:41.354861 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.354841 2562 scope.go:117] "RemoveContainer" containerID="b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc" Jul 07 22:31:41.363284 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.363263 2562 scope.go:117] "RemoveContainer" containerID="5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48" Jul 07 22:31:41.363673 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:31:41.363646 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48\": container with ID starting with 5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48 not found: ID does not exist" containerID="5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48" Jul 07 22:31:41.363673 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.363676 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48"} err="failed to get container status \"5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48\": rpc error: code = NotFound desc = could not find container \"5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48\": container with ID starting with 5366db4e2f11ff6c9cf5bfe7f6d9115e0c6412d564b5afb34cc77a4240f6aa48 not found: ID does not exist" Jul 07 22:31:41.363673 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.363699 2562 scope.go:117] "RemoveContainer" containerID="b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc" Jul 07 22:31:41.364072 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:31:41.364034 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc\": container with ID starting with b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc not found: ID does not exist" containerID="b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc" Jul 07 22:31:41.364165 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.364078 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc"} err="failed to get container status \"b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc\": rpc error: code = NotFound desc = could not find container \"b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc\": container with ID starting with b0931ec0c7d0ea897dd9f1c72d307826787ec991ba9b1d801c4f0a8a5990e5cc not found: ID does not exist" Jul 07 22:31:41.366174 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.366149 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb"] Jul 07 22:31:41.372266 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.372235 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cd9893-openshift-default-6cffc6b9-bmldb"] Jul 07 22:31:41.959975 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:41.959929 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" path="/var/lib/kubelet/pods/12e58bb2-eb12-43e3-86ab-6feefe18a325/volumes" Jul 07 22:31:42.158762 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:42.158723 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:31:42.449416 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:42.449383 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp"] Jul 07 22:31:42.449648 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:42.449623 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" containerID="cri-o://8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3" gracePeriod=30 Jul 07 22:31:43.330557 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330520 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97"] Jul 07 22:31:43.331042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330847 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" Jul 07 22:31:43.331042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330859 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" Jul 07 22:31:43.331042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330866 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" Jul 07 22:31:43.331042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330871 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" Jul 07 22:31:43.331042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330919 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" Jul 07 22:31:43.331042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.330927 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e58bb2-eb12-43e3-86ab-6feefe18a325" containerName="istio-proxy" Jul 07 22:31:43.335184 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.335159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.337380 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.337357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-fa42f6-openshift-default-dockercfg-qjmlf\"" Jul 07 22:31:43.346652 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.346623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97"] Jul 07 22:31:43.532380 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-envoy\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532380 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6da16e5-4837-4547-9531-e99dff33d479-istio-podinfo\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532413 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-certs\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532440 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6da16e5-4837-4547-9531-e99dff33d479-istiod-ca-cert\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-istio-token\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-socket\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-data\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nph67\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-kube-api-access-nph67\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.532792 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.532641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-credential-socket\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.633784 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6da16e5-4837-4547-9531-e99dff33d479-istio-podinfo\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.633784 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-certs\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.633784 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6da16e5-4837-4547-9531-e99dff33d479-istiod-ca-cert\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.633784 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-istio-token\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-socket\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-data\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.633997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nph67\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-kube-api-access-nph67\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-credential-socket\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-envoy\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634422 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-certs\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634422 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634297 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-socket\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634543 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-data\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634543 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634497 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-credential-socket\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.634662 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.634639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6da16e5-4837-4547-9531-e99dff33d479-istiod-ca-cert\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.636503 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.636479 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-envoy\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.636674 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.636657 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6da16e5-4837-4547-9531-e99dff33d479-istio-podinfo\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.642203 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.642168 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-istio-token\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.642385 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.642268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nph67\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-kube-api-access-nph67\") pod \"e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.646123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.646084 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:43.799707 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.799674 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97"] Jul 07 22:31:43.802341 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:31:43.802295 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6da16e5_4837_4547_9531_e99dff33d479.slice/crio-efa0429d7aea13cd6dc943393dffe733d1e49dd761f0afe521fe2b089dbe077b WatchSource:0}: Error finding container efa0429d7aea13cd6dc943393dffe733d1e49dd761f0afe521fe2b089dbe077b: Status 404 returned error can't find the container with id efa0429d7aea13cd6dc943393dffe733d1e49dd761f0afe521fe2b089dbe077b Jul 07 22:31:43.804532 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.804495 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:31:43.804644 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.804572 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:31:43.804644 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:43.804601 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:31:44.187420 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.187379 2562 patch_prober.go:28] interesting pod/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 22:31:44.187615 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.187439 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 22:31:44.361223 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.361186 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" event={"ID":"c6da16e5-4837-4547-9531-e99dff33d479","Type":"ContainerStarted","Data":"8d6f90d2513098575926e88f6c2ea6a87837b441135957549bed9b3fa16bf400"} Jul 07 22:31:44.361639 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.361238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" event={"ID":"c6da16e5-4837-4547-9531-e99dff33d479","Type":"ContainerStarted","Data":"efa0429d7aea13cd6dc943393dffe733d1e49dd761f0afe521fe2b089dbe077b"} Jul 07 22:31:44.381118 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.381064 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podStartSLOduration=1.381049936 podStartE2EDuration="1.381049936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:31:44.379297351 +0000 UTC m=+1982.974056508" watchObservedRunningTime="2026-07-07 22:31:44.381049936 +0000 UTC m=+1982.975809092" Jul 07 22:31:44.647171 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.647137 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:44.652293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:44.652257 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:45.365863 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:45.365829 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:45.366791 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:45.366770 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:31:47.705805 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.705770 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:31:47.768521 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768430 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-data\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768521 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768488 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-certs\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768532 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-podinfo\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768558 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-token\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768594 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-socket\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768612 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-credential-socket\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768634 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xczsn\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-kube-api-access-xczsn\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768670 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-envoy\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.768816 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768692 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istiod-ca-cert\") pod \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\" (UID: \"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33\") " Jul 07 22:31:47.769116 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768816 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:47.769116 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768951 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:47.769116 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.768974 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:47.769116 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.769008 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.769311 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.769131 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:31:47.769403 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.769359 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-data" (OuterVolumeSpecName: "istio-data") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:47.771121 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.771086 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-token" (OuterVolumeSpecName: "istio-token") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:31:47.771240 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.771136 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-kube-api-access-xczsn" (OuterVolumeSpecName: "kube-api-access-xczsn") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "kube-api-access-xczsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:31:47.771240 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.771179 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:31:47.771311 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.771236 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" (UID: "45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:47.869766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869722 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.869766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869753 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.869766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869763 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.869766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869771 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.869766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869780 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xczsn\" (UniqueName: \"kubernetes.io/projected/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-kube-api-access-xczsn\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.870074 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869789 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.870074 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869797 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:47.870074 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:47.869804 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:31:48.380884 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.380849 2562 generic.go:356] "Generic (PLEG): container finished" podID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerID="8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3" exitCode=0 Jul 07 22:31:48.381053 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.380903 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" Jul 07 22:31:48.381053 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.380925 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" event={"ID":"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33","Type":"ContainerDied","Data":"8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3"} Jul 07 22:31:48.381053 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.380973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp" event={"ID":"45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33","Type":"ContainerDied","Data":"4fe6407865da10fca6849c830558b1498fffcb7c36eda74c9eac633a23f36a2f"} Jul 07 22:31:48.381053 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.380994 2562 scope.go:117] "RemoveContainer" containerID="8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3" Jul 07 22:31:48.390285 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.390262 2562 scope.go:117] "RemoveContainer" containerID="9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe" Jul 07 22:31:48.397831 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.397801 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp"] Jul 07 22:31:48.399551 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.399519 2562 scope.go:117] "RemoveContainer" containerID="8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3" Jul 07 22:31:48.400043 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:31:48.400014 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3\": container with ID starting with 8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3 not found: ID does not exist" containerID="8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3" Jul 07 22:31:48.400146 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.400057 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3"} err="failed to get container status \"8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3\": rpc error: code = NotFound desc = could not find container \"8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3\": container with ID starting with 8bdb20a0b4a9958881410d12769cf5baf718229da2fa81fe717249f62dc113b3 not found: ID does not exist" Jul 07 22:31:48.400146 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.400084 2562 scope.go:117] "RemoveContainer" containerID="9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe" Jul 07 22:31:48.400506 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:31:48.400477 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe\": container with ID starting with 9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe not found: ID does not exist" containerID="9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe" Jul 07 22:31:48.400722 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.400516 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe"} err="failed to get container status \"9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe\": rpc error: code = NotFound desc = could not find container \"9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe\": container with ID starting with 9c92849dbf38bef734f47140fd9f2a24b1bfaa0e82af5cbef3781e142059cffe not found: ID does not exist" Jul 07 22:31:48.401555 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:48.401535 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6b32-openshift-default-7f874cbd4c-qjvnp"] Jul 07 22:31:49.954270 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:49.954230 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" path="/var/lib/kubelet/pods/45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33/volumes" Jul 07 22:31:51.539813 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.539775 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb"] Jul 07 22:31:51.540290 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.540274 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" Jul 07 22:31:51.540385 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.540294 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" Jul 07 22:31:51.540427 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.540411 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" Jul 07 22:31:51.540427 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.540422 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" Jul 07 22:31:51.545203 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.545178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.547571 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.547542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-226f1f-openshift-default-dockercfg-8wfxr\"" Jul 07 22:31:51.555178 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.554938 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb"] Jul 07 22:31:51.602265 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-envoy\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602265 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-socket\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602572 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-istio-token\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602572 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de1b331-dd78-46a2-8573-da6d563b5887-istiod-ca-cert\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602572 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzztz\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-kube-api-access-vzztz\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602572 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602527 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-certs\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-credential-socket\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-data\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.602767 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.602662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de1b331-dd78-46a2-8573-da6d563b5887-istio-podinfo\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703583 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de1b331-dd78-46a2-8573-da6d563b5887-istiod-ca-cert\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703802 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzztz\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-kube-api-access-vzztz\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703802 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-certs\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703802 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-credential-socket\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703802 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-data\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703802 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de1b331-dd78-46a2-8573-da6d563b5887-istio-podinfo\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.703802 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-envoy\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704093 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-socket\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704093 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.703837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-istio-token\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704343 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.704157 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-certs\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704343 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.704226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-socket\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704343 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.704270 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-data\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704343 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.704304 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-credential-socket\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.704479 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.704383 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de1b331-dd78-46a2-8573-da6d563b5887-istiod-ca-cert\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.706701 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.706668 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-envoy\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.706916 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.706892 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de1b331-dd78-46a2-8573-da6d563b5887-istio-podinfo\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.711836 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.711806 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-istio-token\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.711970 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.711920 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzztz\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-kube-api-access-vzztz\") pod \"e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:51.856710 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:51.856674 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:52.006671 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.006635 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb"] Jul 07 22:31:52.009878 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:31:52.009845 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de1b331_dd78_46a2_8573_da6d563b5887.slice/crio-770bd35486bc7ace55fa12567e6e18dd8774aa42f59e850cafe3d78f34ef5e3d WatchSource:0}: Error finding container 770bd35486bc7ace55fa12567e6e18dd8774aa42f59e850cafe3d78f34ef5e3d: Status 404 returned error can't find the container with id 770bd35486bc7ace55fa12567e6e18dd8774aa42f59e850cafe3d78f34ef5e3d Jul 07 22:31:52.012228 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.012192 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:31:52.012307 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.012267 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:31:52.012307 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.012296 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:31:52.401361 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.401294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" event={"ID":"7de1b331-dd78-46a2-8573-da6d563b5887","Type":"ContainerStarted","Data":"9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865"} Jul 07 22:31:52.401587 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.401424 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" event={"ID":"7de1b331-dd78-46a2-8573-da6d563b5887","Type":"ContainerStarted","Data":"770bd35486bc7ace55fa12567e6e18dd8774aa42f59e850cafe3d78f34ef5e3d"} Jul 07 22:31:52.422696 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.422640 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podStartSLOduration=1.422623416 podStartE2EDuration="1.422623416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:31:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:31:52.420601601 +0000 UTC m=+1991.015360760" watchObservedRunningTime="2026-07-07 22:31:52.422623416 +0000 UTC m=+1991.017382572" Jul 07 22:31:52.857136 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:52.857099 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:53.862399 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:53.862367 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:54.068693 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:54.068601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:31:54.412138 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:54.412107 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:31:54.413105 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:31:54.413085 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:01.408596 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:01.408545 2562 patch_prober.go:28] interesting pod/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 22:32:01.409101 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:01.408623 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 22:32:02.449521 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:02.449481 2562 generic.go:356] "Generic (PLEG): container finished" podID="c6da16e5-4837-4547-9531-e99dff33d479" containerID="8d6f90d2513098575926e88f6c2ea6a87837b441135957549bed9b3fa16bf400" exitCode=0 Jul 07 22:32:02.449989 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:02.449565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" event={"ID":"c6da16e5-4837-4547-9531-e99dff33d479","Type":"ContainerDied","Data":"8d6f90d2513098575926e88f6c2ea6a87837b441135957549bed9b3fa16bf400"} Jul 07 22:32:02.449989 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:02.449969 2562 scope.go:117] "RemoveContainer" containerID="8d6f90d2513098575926e88f6c2ea6a87837b441135957549bed9b3fa16bf400" Jul 07 22:32:02.451051 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:02.451024 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:02.451141 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:02.451094 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:02.451141 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:02.451133 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:03.456768 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:03.456727 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" event={"ID":"c6da16e5-4837-4547-9531-e99dff33d479","Type":"ContainerStarted","Data":"cc7d6c5431f80a9576acf7876af25a94ab886f32437d19187a3061b8304c6489"} Jul 07 22:32:03.647139 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:03.647096 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:32:03.647365 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:03.647202 2562 patch_prober.go:28] interesting pod/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 07 22:32:03.647365 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:03.647238 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 07 22:32:04.647049 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:04.647003 2562 patch_prober.go:28] interesting pod/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 07 22:32:04.647515 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:04.647060 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 07 22:32:05.646869 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:05.646824 2562 patch_prober.go:28] interesting pod/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 07 22:32:05.647128 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:05.646884 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 07 22:32:06.647282 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:06.647236 2562 patch_prober.go:28] interesting pod/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 07 22:32:06.647806 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:06.647290 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 07 22:32:07.650276 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:07.650242 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:32:07.650693 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:07.650639 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:32:07.651562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:07.651540 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:32:10.241378 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.241183 2562 patch_prober.go:28] interesting pod/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:48384->10.133.0.33:15021: read: connection reset by peer" start-of-body= Jul 07 22:32:10.241378 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.241251 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:48384->10.133.0.33:15021: read: connection reset by peer" Jul 07 22:32:10.495414 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.495282 2562 generic.go:356] "Generic (PLEG): container finished" podID="7de1b331-dd78-46a2-8573-da6d563b5887" containerID="9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865" exitCode=0 Jul 07 22:32:10.495414 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.495366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" event={"ID":"7de1b331-dd78-46a2-8573-da6d563b5887","Type":"ContainerDied","Data":"9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865"} Jul 07 22:32:10.495837 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.495819 2562 scope.go:117] "RemoveContainer" containerID="9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865" Jul 07 22:32:10.496860 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.496831 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:10.496949 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.496897 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:10.496949 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:10.496931 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:11.266503 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:11.266457 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:32:11.502078 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:11.502041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" event={"ID":"7de1b331-dd78-46a2-8573-da6d563b5887","Type":"ContainerStarted","Data":"74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1"} Jul 07 22:32:11.857487 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:11.857444 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:11.857709 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:11.857683 2562 patch_prober.go:28] interesting pod/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 07 22:32:11.857786 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:11.857748 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 07 22:32:12.857600 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:12.857562 2562 patch_prober.go:28] interesting pod/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 07 22:32:12.858214 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:12.858175 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 07 22:32:13.395436 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:13.395386 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97"] Jul 07 22:32:13.395679 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:13.395647 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" containerID="cri-o://cc7d6c5431f80a9576acf7876af25a94ab886f32437d19187a3061b8304c6489" gracePeriod=30 Jul 07 22:32:13.857755 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:13.857711 2562 patch_prober.go:28] interesting pod/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 07 22:32:13.858241 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:13.857778 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 07 22:32:14.857286 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:14.857251 2562 patch_prober.go:28] interesting pod/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 07 22:32:14.857523 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:14.857314 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 07 22:32:15.861345 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:15.861302 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:15.861806 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:15.861591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:15.862425 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:15.862409 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:18.545973 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.545936 2562 generic.go:356] "Generic (PLEG): container finished" podID="c6da16e5-4837-4547-9531-e99dff33d479" containerID="cc7d6c5431f80a9576acf7876af25a94ab886f32437d19187a3061b8304c6489" exitCode=0 Jul 07 22:32:18.546384 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.546023 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" event={"ID":"c6da16e5-4837-4547-9531-e99dff33d479","Type":"ContainerDied","Data":"cc7d6c5431f80a9576acf7876af25a94ab886f32437d19187a3061b8304c6489"} Jul 07 22:32:18.547267 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.546121 2562 scope.go:117] "RemoveContainer" containerID="8d6f90d2513098575926e88f6c2ea6a87837b441135957549bed9b3fa16bf400" Jul 07 22:32:18.653452 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.653417 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:32:18.757588 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757487 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nph67\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-kube-api-access-nph67\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757588 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757539 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6da16e5-4837-4547-9531-e99dff33d479-istio-podinfo\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757588 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757577 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6da16e5-4837-4547-9531-e99dff33d479-istiod-ca-cert\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757866 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757604 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-credential-socket\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757866 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757647 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-envoy\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757866 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757672 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-certs\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757866 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-istio-token\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757866 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757743 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-socket\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.757866 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.757773 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-data\") pod \"c6da16e5-4837-4547-9531-e99dff33d479\" (UID: \"c6da16e5-4837-4547-9531-e99dff33d479\") " Jul 07 22:32:18.758164 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.758024 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:18.758164 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.758042 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:18.758164 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.758118 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6da16e5-4837-4547-9531-e99dff33d479-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:32:18.758320 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.758239 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:18.758760 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.758731 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:18.759984 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.759953 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:32:18.760086 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.759989 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-kube-api-access-nph67" (OuterVolumeSpecName: "kube-api-access-nph67") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "kube-api-access-nph67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:32:18.760558 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.760541 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:18.760644 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.760628 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6da16e5-4837-4547-9531-e99dff33d479-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6da16e5-4837-4547-9531-e99dff33d479" (UID: "c6da16e5-4837-4547-9531-e99dff33d479"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:32:18.858810 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858779 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.858810 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858807 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.858810 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858816 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nph67\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-kube-api-access-nph67\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.859045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858827 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6da16e5-4837-4547-9531-e99dff33d479-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.859045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858835 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6da16e5-4837-4547-9531-e99dff33d479-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.859045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858842 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.859045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858853 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.859045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858861 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6da16e5-4837-4547-9531-e99dff33d479-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:18.859045 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:18.858881 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6da16e5-4837-4547-9531-e99dff33d479-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:19.552125 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:19.552094 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" Jul 07 22:32:19.552606 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:19.552121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97" event={"ID":"c6da16e5-4837-4547-9531-e99dff33d479","Type":"ContainerDied","Data":"efa0429d7aea13cd6dc943393dffe733d1e49dd761f0afe521fe2b089dbe077b"} Jul 07 22:32:19.552606 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:19.552187 2562 scope.go:117] "RemoveContainer" containerID="cc7d6c5431f80a9576acf7876af25a94ab886f32437d19187a3061b8304c6489" Jul 07 22:32:19.574625 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:19.574591 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97"] Jul 07 22:32:19.577997 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:19.577971 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-fa42f6-openshift-default-55546c4485-xxx97"] Jul 07 22:32:19.954612 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:19.954576 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6da16e5-4837-4547-9531-e99dff33d479" path="/var/lib/kubelet/pods/c6da16e5-4837-4547-9531-e99dff33d479/volumes" Jul 07 22:32:20.630081 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:20.630047 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb"] Jul 07 22:32:20.630539 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:20.630263 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" containerID="cri-o://74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1" gracePeriod=30 Jul 07 22:32:24.868266 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:24.868225 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:32:25.898017 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:25.897936 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:26.029059 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029022 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-istio-token\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029059 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029065 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-certs\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029425 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029096 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-socket\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029425 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029117 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-credential-socket\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029425 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029184 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de1b331-dd78-46a2-8573-da6d563b5887-istio-podinfo\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029425 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029244 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de1b331-dd78-46a2-8573-da6d563b5887-istiod-ca-cert\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029425 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029279 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-data\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.029691 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029418 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:26.029691 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029433 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:26.029691 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029580 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:26.029691 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029666 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de1b331-dd78-46a2-8573-da6d563b5887-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:32:26.029945 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029918 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzztz\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-kube-api-access-vzztz\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.030055 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.029963 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-envoy\") pod \"7de1b331-dd78-46a2-8573-da6d563b5887\" (UID: \"7de1b331-dd78-46a2-8573-da6d563b5887\") " Jul 07 22:32:26.030244 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.030227 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de1b331-dd78-46a2-8573-da6d563b5887-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.030293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.030248 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.030293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.030260 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.030293 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.030272 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.030500 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.030324 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-data" (OuterVolumeSpecName: "istio-data") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:26.031793 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.031761 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7de1b331-dd78-46a2-8573-da6d563b5887-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:32:26.031793 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.031764 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-istio-token" (OuterVolumeSpecName: "istio-token") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:32:26.032073 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.032053 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-kube-api-access-vzztz" (OuterVolumeSpecName: "kube-api-access-vzztz") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "kube-api-access-vzztz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:32:26.032357 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.032313 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7de1b331-dd78-46a2-8573-da6d563b5887" (UID: "7de1b331-dd78-46a2-8573-da6d563b5887"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:32:26.130718 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.130681 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de1b331-dd78-46a2-8573-da6d563b5887-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.130718 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.130714 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.130718 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.130723 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzztz\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-kube-api-access-vzztz\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.130975 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.130733 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de1b331-dd78-46a2-8573-da6d563b5887-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.130975 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.130744 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de1b331-dd78-46a2-8573-da6d563b5887-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:32:26.587820 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.587779 2562 generic.go:356] "Generic (PLEG): container finished" podID="7de1b331-dd78-46a2-8573-da6d563b5887" containerID="74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1" exitCode=0 Jul 07 22:32:26.588037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.587838 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" Jul 07 22:32:26.588037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.587849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" event={"ID":"7de1b331-dd78-46a2-8573-da6d563b5887","Type":"ContainerDied","Data":"74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1"} Jul 07 22:32:26.588037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.587899 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb" event={"ID":"7de1b331-dd78-46a2-8573-da6d563b5887","Type":"ContainerDied","Data":"770bd35486bc7ace55fa12567e6e18dd8774aa42f59e850cafe3d78f34ef5e3d"} Jul 07 22:32:26.588037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.587921 2562 scope.go:117] "RemoveContainer" containerID="74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1" Jul 07 22:32:26.597190 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.597165 2562 scope.go:117] "RemoveContainer" containerID="9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865" Jul 07 22:32:26.605967 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.605945 2562 scope.go:117] "RemoveContainer" containerID="74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1" Jul 07 22:32:26.606515 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:32:26.606485 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1\": container with ID starting with 74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1 not found: ID does not exist" containerID="74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1" Jul 07 22:32:26.606661 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.606527 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1"} err="failed to get container status \"74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1\": rpc error: code = NotFound desc = could not find container \"74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1\": container with ID starting with 74a790d58aad27c3df24df9a60abd471d360b2c7ac796a6d07be187d442656f1 not found: ID does not exist" Jul 07 22:32:26.606661 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.606559 2562 scope.go:117] "RemoveContainer" containerID="9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865" Jul 07 22:32:26.606884 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:32:26.606861 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865\": container with ID starting with 9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865 not found: ID does not exist" containerID="9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865" Jul 07 22:32:26.606943 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.606888 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865"} err="failed to get container status \"9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865\": rpc error: code = NotFound desc = could not find container \"9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865\": container with ID starting with 9444b5b9c7ac8048404f41f0044200d83b1199c516bd704e70b48ba3cc601865 not found: ID does not exist" Jul 07 22:32:26.608322 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.608303 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb"] Jul 07 22:32:26.612375 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:26.612344 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-226f1f-openshift-default-75bfc866f5-4spnb"] Jul 07 22:32:27.954254 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:27.954222 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" path="/var/lib/kubelet/pods/7de1b331-dd78-46a2-8573-da6d563b5887/volumes" Jul 07 22:32:29.430433 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430389 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75"] Jul 07 22:32:29.430912 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430860 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" Jul 07 22:32:29.430912 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430879 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a2cb3c-dc9d-4bb9-b7cb-d2a9d4d04a33" containerName="istio-proxy" Jul 07 22:32:29.430912 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430891 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" Jul 07 22:32:29.430912 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430900 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" Jul 07 22:32:29.431123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430925 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" Jul 07 22:32:29.431123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.430933 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" Jul 07 22:32:29.431123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.431015 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" Jul 07 22:32:29.431123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.431028 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" Jul 07 22:32:29.431123 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.431039 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" Jul 07 22:32:29.433133 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.433108 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.435530 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.435495 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-5b00ce-openshift-default-dockercfg-fg726\"" Jul 07 22:32:29.446562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.446534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75"] Jul 07 22:32:29.568281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-socket\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568281 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568276 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-certs\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-podinfo\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npvt8\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-kube-api-access-npvt8\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-credential-socket\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568683 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568546 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-envoy\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568683 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istiod-ca-cert\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568683 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-data\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.568821 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.568700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-token\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.669864 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.669826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istiod-ca-cert\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.669864 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.669867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-data\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.669883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-token\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.669908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-socket\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.669932 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-certs\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.669993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-podinfo\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670030 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npvt8\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-kube-api-access-npvt8\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-credential-socket\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-envoy\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670536 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670490 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-data\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670596 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670567 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-credential-socket\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670790 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670764 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-socket\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670775 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istiod-ca-cert\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.670920 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.670889 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-certs\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.673037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.673016 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-envoy\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.673365 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.673322 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-podinfo\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.677584 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.677557 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-token\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.677678 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.677636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npvt8\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-kube-api-access-npvt8\") pod \"e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.745941 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.745850 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:29.914608 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.914491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75"] Jul 07 22:32:29.916940 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:32:29.916908 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07a964e_4c8f_4732_af29_a1b4a73ea4b8.slice/crio-d8a5c05d9c35b00538ce6ef10ee3fa7a796c60ea961f058c1d99e478cef2d733 WatchSource:0}: Error finding container d8a5c05d9c35b00538ce6ef10ee3fa7a796c60ea961f058c1d99e478cef2d733: Status 404 returned error can't find the container with id d8a5c05d9c35b00538ce6ef10ee3fa7a796c60ea961f058c1d99e478cef2d733 Jul 07 22:32:29.919042 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.919010 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:29.919136 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.919112 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:29.919200 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:29.919157 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:30.611302 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:30.611269 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" event={"ID":"b07a964e-4c8f-4732-af29-a1b4a73ea4b8","Type":"ContainerStarted","Data":"54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e"} Jul 07 22:32:30.611719 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:30.611318 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" event={"ID":"b07a964e-4c8f-4732-af29-a1b4a73ea4b8","Type":"ContainerStarted","Data":"d8a5c05d9c35b00538ce6ef10ee3fa7a796c60ea961f058c1d99e478cef2d733"} Jul 07 22:32:30.632208 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:30.632154 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podStartSLOduration=1.632137208 podStartE2EDuration="1.632137208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:32:30.630377185 +0000 UTC m=+2029.225136340" watchObservedRunningTime="2026-07-07 22:32:30.632137208 +0000 UTC m=+2029.226896426" Jul 07 22:32:30.746442 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:30.746399 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:30.752218 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:30.752190 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:31.616652 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:31.616618 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:31.617818 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:31.617798 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:40.058731 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:40.058694 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:32:47.659562 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:47.659513 2562 patch_prober.go:28] interesting pod/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 22:32:47.659978 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:47.659584 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 22:32:48.698342 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:48.698294 2562 generic.go:356] "Generic (PLEG): container finished" podID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerID="54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e" exitCode=0 Jul 07 22:32:48.698902 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:48.698375 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" event={"ID":"b07a964e-4c8f-4732-af29-a1b4a73ea4b8","Type":"ContainerDied","Data":"54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e"} Jul 07 22:32:48.698976 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:48.698911 2562 scope.go:117] "RemoveContainer" containerID="54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e" Jul 07 22:32:48.700027 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:48.699997 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:48.700097 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:48.700065 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:48.700133 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:48.700099 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 22:32:49.705988 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:49.705944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" event={"ID":"b07a964e-4c8f-4732-af29-a1b4a73ea4b8","Type":"ContainerStarted","Data":"2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d"} Jul 07 22:32:49.746147 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:49.746109 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:49.746498 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:49.746461 2562 patch_prober.go:28] interesting pod/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 07 22:32:49.746638 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:49.746526 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 07 22:32:50.746992 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:50.746953 2562 patch_prober.go:28] interesting pod/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 07 22:32:50.747438 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:50.747014 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 07 22:32:51.746629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:51.746584 2562 patch_prober.go:28] interesting pod/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 07 22:32:51.746858 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:51.746651 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 07 22:32:52.746800 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:52.746750 2562 patch_prober.go:28] interesting pod/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 07 22:32:52.747221 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:52.746832 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 07 22:32:53.750760 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:53.750725 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:53.751165 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:53.750996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:32:53.751996 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:32:53.751972 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:35:45.862825 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:35:45.862778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:35:54.942167 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:35:54.942130 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75"] Jul 07 22:35:54.942713 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:35:54.942389 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" containerID="cri-o://2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d" gracePeriod=30 Jul 07 22:35:57.163315 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:35:57.163278 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-whwx8"] Jul 07 22:36:00.215109 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.215025 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:36:00.304438 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304394 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istiod-ca-cert\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304457 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-envoy\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304488 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-certs\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304503 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-data\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304521 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-token\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304539 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-credential-socket\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304567 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-podinfo\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304593 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-socket\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.304629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304617 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npvt8\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-kube-api-access-npvt8\") pod \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\" (UID: \"b07a964e-4c8f-4732-af29-a1b4a73ea4b8\") " Jul 07 22:36:00.305037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304906 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:00.305037 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.304967 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:36:00.305159 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.305130 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:00.305250 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.305225 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:00.305792 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.305764 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:00.306994 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.306970 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:36:00.307079 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.307021 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-kube-api-access-npvt8" (OuterVolumeSpecName: "kube-api-access-npvt8") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "kube-api-access-npvt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:36:00.307432 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.307405 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:00.307484 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.307432 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b07a964e-4c8f-4732-af29-a1b4a73ea4b8" (UID: "b07a964e-4c8f-4732-af29-a1b4a73ea4b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:36:00.405743 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405685 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-podinfo\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.405743 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405732 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.405743 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405745 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npvt8\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-kube-api-access-npvt8\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.405743 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405758 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istiod-ca-cert\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.406031 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405771 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-envoy\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.406031 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405783 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-workload-certs\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.406031 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405793 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-data\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.406031 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405803 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-istio-token\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.406031 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.405812 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07a964e-4c8f-4732-af29-a1b4a73ea4b8-credential-socket\") on node \"ip-10-0-132-125.ec2.internal\" DevicePath \"\"" Jul 07 22:36:00.531182 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.531092 2562 generic.go:356] "Generic (PLEG): container finished" podID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerID="2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d" exitCode=0 Jul 07 22:36:00.531182 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.531140 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" Jul 07 22:36:00.531182 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.531170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" event={"ID":"b07a964e-4c8f-4732-af29-a1b4a73ea4b8","Type":"ContainerDied","Data":"2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d"} Jul 07 22:36:00.531452 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.531243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75" event={"ID":"b07a964e-4c8f-4732-af29-a1b4a73ea4b8","Type":"ContainerDied","Data":"d8a5c05d9c35b00538ce6ef10ee3fa7a796c60ea961f058c1d99e478cef2d733"} Jul 07 22:36:00.531452 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.531267 2562 scope.go:117] "RemoveContainer" containerID="2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d" Jul 07 22:36:00.540523 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.540501 2562 scope.go:117] "RemoveContainer" containerID="54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e" Jul 07 22:36:00.549028 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.549004 2562 scope.go:117] "RemoveContainer" containerID="2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d" Jul 07 22:36:00.549385 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:36:00.549360 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d\": container with ID starting with 2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d not found: ID does not exist" containerID="2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d" Jul 07 22:36:00.549445 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.549398 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d"} err="failed to get container status \"2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d\": rpc error: code = NotFound desc = could not find container \"2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d\": container with ID starting with 2bfe3a08ce01d10a8622c2d53d2144985283bac9d200fcef7e88908c7982c82d not found: ID does not exist" Jul 07 22:36:00.549445 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.549426 2562 scope.go:117] "RemoveContainer" containerID="54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e" Jul 07 22:36:00.549691 ip-10-0-132-125 kubenswrapper[2562]: E0707 22:36:00.549675 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e\": container with ID starting with 54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e not found: ID does not exist" containerID="54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e" Jul 07 22:36:00.549754 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.549692 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e"} err="failed to get container status \"54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e\": rpc error: code = NotFound desc = could not find container \"54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e\": container with ID starting with 54ad57548c6c5af4be81d3b5bc5f7e59673a6f1aa63d2f6e044aa2f5e208f74e not found: ID does not exist" Jul 07 22:36:00.552942 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.552911 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75"] Jul 07 22:36:00.558623 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:00.558594 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-5b00ce-openshift-default-7b66455f4c-wqh75"] Jul 07 22:36:01.088629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:01.088604 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dd55458bc-87cpx_87033ff9-b40d-452e-945a-07c288ac9c05/copy-manifests/0.log" Jul 07 22:36:01.095452 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:01.095421 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dd55458bc-87cpx_87033ff9-b40d-452e-945a-07c288ac9c05/manager/0.log" Jul 07 22:36:01.955433 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:01.955401 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" path="/var/lib/kubelet/pods/b07a964e-4c8f-4732-af29-a1b4a73ea4b8/volumes" Jul 07 22:36:03.418577 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:03.418500 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-whwx8_1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c/limitador/0.log" Jul 07 22:36:03.894377 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:03.894346 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgk2r_cb36678c-afb0-4022-bbcf-38970b987150/istio-proxy/0.log" Jul 07 22:36:04.019629 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:04.019595 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n7jhn_bb1e4480-a950-4536-a8f0-e60886b92771/discovery/0.log" Jul 07 22:36:05.568000 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:05.567968 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lr9ts_31f693a8-0493-4dfd-8629-5a9d7fde6021/main/0.log" Jul 07 22:36:12.870898 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:12.870861 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l6chz_9eba0b17-6bdb-4a42-ad6e-ba0d705772ef/global-pull-secret-syncer/0.log" Jul 07 22:36:12.982835 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:12.982808 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zfb55_13e9d2e6-45f5-4e22-99b9-c8ce050e23b0/konnectivity-agent/0.log" Jul 07 22:36:13.052151 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:13.052124 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-125.ec2.internal_74f5d17caa4e3653403abe6e004ae3b2/haproxy/0.log" Jul 07 22:36:18.720962 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:18.720922 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-whwx8_1c7dbe0d-88aa-4cbf-bab1-f5200b2b328c/limitador/0.log" Jul 07 22:36:20.592935 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:20.592906 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-ct22z_494370d8-286d-4c39-9b7c-15995a13779d/cluster-monitoring-operator/0.log" Jul 07 22:36:20.966272 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:20.966191 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ff69k_6a5edc49-24aa-430e-aceb-8fd4b52b2d33/node-exporter/0.log" Jul 07 22:36:20.986749 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:20.986720 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ff69k_6a5edc49-24aa-430e-aceb-8fd4b52b2d33/kube-rbac-proxy/0.log" Jul 07 22:36:21.009317 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:21.009264 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ff69k_6a5edc49-24aa-430e-aceb-8fd4b52b2d33/init-textfile/0.log" Jul 07 22:36:22.844987 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:22.844959 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-vbppw_f1f8ed4f-3ffb-4dd4-9a6f-f793b373cf6c/networking-console-plugin/0.log" Jul 07 22:36:23.330516 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330482 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm"] Jul 07 22:36:23.330791 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330779 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" Jul 07 22:36:23.330838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330793 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" Jul 07 22:36:23.330838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330810 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" Jul 07 22:36:23.330838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330815 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" Jul 07 22:36:23.330838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330826 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" Jul 07 22:36:23.330838 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330832 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de1b331-dd78-46a2-8573-da6d563b5887" containerName="istio-proxy" Jul 07 22:36:23.331007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330841 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" Jul 07 22:36:23.331007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330846 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" Jul 07 22:36:23.331007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330900 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6da16e5-4837-4547-9531-e99dff33d479" containerName="istio-proxy" Jul 07 22:36:23.331007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330909 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" Jul 07 22:36:23.331007 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.330915 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b07a964e-4c8f-4732-af29-a1b4a73ea4b8" containerName="istio-proxy" Jul 07 22:36:23.333717 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.333700 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.336075 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.336050 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wh6z7\"/\"default-dockercfg-bhddq\"" Jul 07 22:36:23.336212 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.336057 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wh6z7\"/\"openshift-service-ca.crt\"" Jul 07 22:36:23.336914 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.336898 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wh6z7\"/\"kube-root-ca.crt\"" Jul 07 22:36:23.340350 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.340309 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm"] Jul 07 22:36:23.506101 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.506063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-585l9\" (UniqueName: \"kubernetes.io/projected/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-kube-api-access-585l9\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.506289 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.506114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-podres\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.506289 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.506180 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-lib-modules\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.506289 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.506211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-sys\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.506289 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.506250 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-proc\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607582 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-585l9\" (UniqueName: \"kubernetes.io/projected/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-kube-api-access-585l9\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607582 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-podres\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607582 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-lib-modules\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607582 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-sys\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607867 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-proc\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607867 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607698 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-sys\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607867 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607698 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-proc\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607867 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607747 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-lib-modules\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.607867 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.607767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-podres\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.617106 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.617066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-585l9\" (UniqueName: \"kubernetes.io/projected/240d0cbc-8fe4-49b5-b360-ebed4a7b7171-kube-api-access-585l9\") pod \"perf-node-gather-daemonset-95pkm\" (UID: \"240d0cbc-8fe4-49b5-b360-ebed4a7b7171\") " pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.646245 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.646206 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:23.787386 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:23.787317 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm"] Jul 07 22:36:23.790617 ip-10-0-132-125 kubenswrapper[2562]: W0707 22:36:23.790575 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod240d0cbc_8fe4_49b5_b360_ebed4a7b7171.slice/crio-5f4e6e1e480a3dd50f50e40afd1de5de33430e08ee11cb56fdfff166562f915c WatchSource:0}: Error finding container 5f4e6e1e480a3dd50f50e40afd1de5de33430e08ee11cb56fdfff166562f915c: Status 404 returned error can't find the container with id 5f4e6e1e480a3dd50f50e40afd1de5de33430e08ee11cb56fdfff166562f915c Jul 07 22:36:24.629812 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:24.629771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" event={"ID":"240d0cbc-8fe4-49b5-b360-ebed4a7b7171","Type":"ContainerStarted","Data":"5cccbbe0f295c28c93ed9518393eb6c25c398c2e295ac385b68aa1e9b8b3603d"} Jul 07 22:36:24.630305 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:24.629830 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" event={"ID":"240d0cbc-8fe4-49b5-b360-ebed4a7b7171","Type":"ContainerStarted","Data":"5f4e6e1e480a3dd50f50e40afd1de5de33430e08ee11cb56fdfff166562f915c"} Jul 07 22:36:24.630305 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:24.629892 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:24.645013 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:24.644962 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" podStartSLOduration=1.644948036 podStartE2EDuration="1.644948036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:36:24.644028013 +0000 UTC m=+2263.238787173" watchObservedRunningTime="2026-07-07 22:36:24.644948036 +0000 UTC m=+2263.239707192" Jul 07 22:36:24.760660 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:24.760635 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-j7fhc_d3a3a117-7d53-47c5-b30e-f9ab475065eb/volume-data-source-validator/0.log" Jul 07 22:36:25.858310 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:25.858282 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l4ptc_24de3685-1707-4673-9b91-33499812966c/dns/0.log" Jul 07 22:36:25.877445 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:25.877420 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l4ptc_24de3685-1707-4673-9b91-33499812966c/kube-rbac-proxy/0.log" Jul 07 22:36:26.000080 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:26.000053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vp854_6dcab854-eee8-4d5d-a448-602298a4e71d/dns-node-resolver/0.log" Jul 07 22:36:26.602940 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:26.602912 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m9qs8_d51c526d-3e94-4c65-821c-ae2b5466a4bb/node-ca/0.log" Jul 07 22:36:27.552073 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:27.552043 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgk2r_cb36678c-afb0-4022-bbcf-38970b987150/istio-proxy/0.log" Jul 07 22:36:27.649065 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:27.649023 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n7jhn_bb1e4480-a950-4536-a8f0-e60886b92771/discovery/0.log" Jul 07 22:36:28.620737 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:28.620707 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tmvwz_259d2c92-3c9b-4475-8693-e0eb6bd3969c/serve-healthcheck-canary/0.log" Jul 07 22:36:29.262936 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:29.262903 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lk65s_9012fc19-7a26-46f0-8193-eac305bbf8fd/kube-rbac-proxy/0.log" Jul 07 22:36:29.282427 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:29.282397 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lk65s_9012fc19-7a26-46f0-8193-eac305bbf8fd/exporter/0.log" Jul 07 22:36:29.302673 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:29.302648 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lk65s_9012fc19-7a26-46f0-8193-eac305bbf8fd/extractor/0.log" Jul 07 22:36:30.644481 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:30.644454 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wh6z7/perf-node-gather-daemonset-95pkm" Jul 07 22:36:31.777766 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:31.777738 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dd55458bc-87cpx_87033ff9-b40d-452e-945a-07c288ac9c05/manager/0.log" Jul 07 22:36:31.798284 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:31.798252 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dd55458bc-87cpx_87033ff9-b40d-452e-945a-07c288ac9c05/copy-manifests/0.log" Jul 07 22:36:33.345806 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:33.345775 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c4f8f67f4-9xc9w_f4cb09eb-37d8-4fca-af8d-cf44fbd03e87/manager/0.log" Jul 07 22:36:34.567982 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:34.567954 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgk2r_cb36678c-afb0-4022-bbcf-38970b987150/istio-proxy/0.log" Jul 07 22:36:34.660693 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:34.660658 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n7jhn_bb1e4480-a950-4536-a8f0-e60886b92771/discovery/0.log" Jul 07 22:36:36.242498 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:36.242463 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgk2r_cb36678c-afb0-4022-bbcf-38970b987150/istio-proxy/0.log" Jul 07 22:36:36.338696 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:36.338661 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n7jhn_bb1e4480-a950-4536-a8f0-e60886b92771/discovery/0.log" Jul 07 22:36:38.696701 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:38.696668 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-gprw8_a166678e-5c5b-490d-ac16-7f387a4436a0/migrator/0.log" Jul 07 22:36:38.716248 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:38.716218 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-gprw8_a166678e-5c5b-490d-ac16-7f387a4436a0/graceful-termination/0.log" Jul 07 22:36:40.448950 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.448925 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/kube-multus-additional-cni-plugins/0.log" Jul 07 22:36:40.468993 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.468966 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/egress-router-binary-copy/0.log" Jul 07 22:36:40.489806 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.489782 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/cni-plugins/0.log" Jul 07 22:36:40.509981 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.509952 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/bond-cni-plugin/0.log" Jul 07 22:36:40.532760 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.532730 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/routeoverride-cni/0.log" Jul 07 22:36:40.553342 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.553297 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/whereabouts-cni-bincopy/0.log" Jul 07 22:36:40.573796 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.573770 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc8mv_644b681a-6015-4423-889f-120d814f62e0/whereabouts-cni/0.log" Jul 07 22:36:40.605189 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.605157 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qxzqd_07efbfcf-2445-4754-93af-05c5041397b6/kube-multus/0.log" Jul 07 22:36:40.698196 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.698162 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-htffx_a61b1873-a762-4e95-aa01-818198e15cc3/network-metrics-daemon/0.log" Jul 07 22:36:40.715769 ip-10-0-132-125 kubenswrapper[2562]: I0707 22:36:40.715685 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-htffx_a61b1873-a762-4e95-aa01-818198e15cc3/kube-rbac-proxy/0.log"